Computer-readable storage medium, information processing system, and information processing method

ABSTRACT

It is determined, based on attitude data, whether or not a direction of a predetermined axis set in a portable display device is within a first range. When the result of the determination is that the direction of the predetermined axis is within the first range, an action of a first object arranged in a virtual world is controlled based on the direction of the predetermined axis. When it is determined that the direction of the predetermined axis is outside the first range, at least an action of a second object arranged in the virtual world is controlled based on the direction of the predetermined axis. Then, a first image showing the virtual world including at least the first object and the second object is displayed on the portable display device.

CROSS REFERENCE TO RELATED APPLICATION

The disclosure of Japanese Patent Application No. 2011-083455, filed onApr. 5, 2011, is incorporated herein by reference.

FIELD

The exemplary embodiments relate to computer-readable storage media,information processing systems, and information processing methods. Moreparticularly, the exemplary embodiments relate to computer-readablestorage media, information processing systems, and informationprocessing methods, which enable, for example, processing based on anaction of a user.

BACKGROUND AND SUMMARY

Conventionally, there are known game apparatuses in which a single usercontrols a plurality of objects in a virtual world. For example, in aconventional game apparatus, in a virtual world, an action of a playercharacter A, an action of a player character B, a position of lighting,and the like are controlled based on an operation by a user.Specifically, when the user presses a button A, the player character Ajumps in the virtual world. When the user presses a button B, the playercharacter B jumps in the virtual world. When the user presses a crosskey, an area illuminated by light in the virtual world is shifted towarda direction in which the cross key is pressed.

In the conventional game apparatus, however, the single user is requiredto operate the plurality of player characters and the lighting by usingdifferent operation devices, respectively. Accordingly, the user isrequired to perform complicated operation, which may cause erroneousoperation. Further, it may take a while for the user to get used to theoperation.

Accordingly, an object of the exemplary embodiments is to provide acomputer-readable storage medium, an information processing system, andan information processing method, which allows a user to easily performan operation when he/she operates a plurality of objects in a virtualworld.

In order to achieve the above-mentioned object, the exemplaryembodiments have the following configurations. It should be understoodthat the scope of the exemplary embodiments is interpreted only by thescope of the claims. In event of any conflict between the scope of theclaims and the scope of the description in this section, the scope ofthe claims has priority.

An example of a configuration of a computer-readable storage mediumaccording to exemplary embodiments has stored therein an informationprocessing program executed on a computer of an information processingapparatus which is capable of displaying an image on a portable displaydevice. The portable display device outputs at least attitude data basedon an attitude of a body of the portable display device. The informationprocessing program causes the computer to act as a directiondetermination section, a first object action control section, a secondobject action control section, and a display control section. Thedirection determination section determines, based on the attitude data,whether or not a direction of a predetermined axis set in the portabledisplay device is within a first range. The first object action controlsection controls an action of a first object arranged in a virtualworld, in accordance with the direction of the predetermined axis, whenthe result of the determination is that the direction of thepredetermined axis is within the first range. The second object actioncontrol section controls at least an action of a second object arrangedin the virtual world, in accordance with the direction of thepredetermined axis, when the result of the determination is that thedirection of the predetermined axis is outside the first range. Thedisplay control section displays, on the portable display device, afirst image showing the virtual world including at least the firstobject and the second object.

The information processing apparatus may be an apparatus for executing agame process and generating an image based on the game process, or maybe a versatile apparatus such as a general personal computer. Theportable display device may have a size small enough to be carried by auser. Typically, the portable display device may be a display devicewhich allows the user, holding the portable display device with his/herboth hands, to view an image displayed thereon. The portable displaydevice may include other components than the section for outputting atleast attitude data and the section for displaying the first image, likea terminal device of exemplary embodiments described later, or may notinclude other components.

According to the above configuration, a user is allowed to perform anoperation on the first object and the second object, based on theattitude of a single device. Therefore, the user is allowed to easilyperform an operation when he/she operates a plurality of object in thevirtual world.

The information processing program may further causes the computer toact as an attitude calculation section. The attitude calculation sectioncalculates an attitude of the portable display device, based on theattitude data. In this case, the direction determination section maydetermine whether or not the direction of the predetermined axis iswithin the first range, based on the attitude of the portable displaydevice.

According to the above configuration, it is possible to calculate theattitude of the portable display device by using the attitude data, andcontrol the actions of the first and second objects based on theattitude of the portable display device.

The first object action control section may control the action of thefirst object, based on a change in the direction of the predeterminedaxis. The second object action control section may control at least theaction of the second object, based on the change in the direction of thepredetermined axis.

According to the above configuration, the user is allowed to control theaction of the first object and the action of the second object bychanging the attitude of the portable display device.

The first object action control section may control the action of thefirst object such that the greater the change in the direction of thepredetermined axis is, the greater the movement of the first object is.The second object action control section may control the action of thesecond object such that the greater the change in the direction of thepredetermined axis is, the greater the movement of the second object is.The movement of the first object (the movement of the second object) maybe a movement in which the position of the object changes in the virtualworld, or a movement in which the object rotates without changing itsposition.

According to the above configuration, the user is allowed to control anamount of movement (including an amount of rotation) of the first objectand the second object in accordance with the magnitude of change in theattitude of the portable display device.

The first object action control section may control the action of thefirst object by causing the first object to rotate in accordance with anangle of the predetermined axis. The second object action controlsection may control the action of the second object by causing thesecond object to rotate in accordance with the angle of thepredetermined axis.

According to the above configuration, the user is allowed to control theaction of the first object and the action of the second object inaccordance with the angle of the portable display device.

The second object action control section controls the action of thesecond object in accordance with the direction of the predeterminedaxis, and controls the action of the first object in accordance with theaction of the second object.

According to the above configuration, the user is allowed to perform acontrol for causing the first object and the second object to movesimultaneously, by changing the attitude of the portable display devicesuch that the direction of the predetermined axis is outside the firstrange.

The second object action control section may control the action of thesecond object in accordance with the direction of the predeterminedaxis, and control the action of the first object such that the firstobject performs the same action as the second object.

According to the above configuration, the user is allowed to cause thefirst object and the second object to perform the same action, bychanging the attitude of the portable display device such that thedirection of the predetermined axis is outside the first range.

The direction determination section may set, as the first range, a rangewithin a first angle, and sets, as a second range, a range which iswithin a second angle larger than the first angle and is outside thefirst range, and may determine whether or not the direction of thepredetermined axis is within the second range. The second object actioncontrol section may control the action of the second object by causingthe second object to rotate at a first angular velocity in accordancewith the direction of the predetermined axis, when the result of thedetermination is that the direction of the predetermined axis is withinthe second range, and may control the action of the second object bycausing the second object to rotate at a second angular velocity whichis lower than the first angular velocity, in accordance with thedirection of the predetermined axis, when the result of thedetermination is that the direction of the predetermined axis is neitherin the first range nor in the second range.

According to the above configuration, the speed at which the secondobject rotates in the virtual world can be controlled based on theattitude of the portable display device.

The direction determination section may set a reference direction withrespect to the direction of the predetermined axis, and set, as thefirst range, a range within the first angle centering around thereference direction, and set, as the second range, a range which iswithin the second angle centering around the reference direction and isoutside the first range.

According to the above configuration, the speed at which the secondobject rotates in the virtual world can be controlled based on theattitude of the portable display device with respect to the referencedirection.

The direction determination section may set a reference direction withrespect to the direction of the predetermined axis, and set, as thefirst range, a range within the first angle centering around thereference direction.

According to the above configuration, the action of the first object iscontrolled by changing the attitude of the portable display device suchthat the direction of the predetermined axis set in the portable displaydevice is within a range relatively close to the reference direction,and the action of the second object is controlled by changing theattitude of the portable display device such that the direction of thepredetermined axis is within a range relatively distant from thereference direction. Accordingly, the user is allowed to selectively usethe operation for controlling the action of the first object and theoperation for controlling the action of the second object, depending onan amount of change in the attitude of the portable display device fromthe reference direction.

The direction determination section may set the reference direction on apredetermined horizontal plane, and set, as the first range, a rangewithin the first angle on the horizontal plane centering around thereference direction.

According to the above configuration, the user is allowed to selectivelyuse the operation for controlling the action of the first object and theoperation for controlling the action of the second object, depending onan amount of change in the attitude of the portable display device fromthe reference direction along the predetermined plane.

The display control section may display, separately from the firstimage, a second image showing the virtual world on another displaydevice connected to the information processing apparatus.

The other display device is a display device connected to theinformation processing apparatus, such as a monitor 2 described later.Any type of display device may be used so long as it is separated fromthe portable display device, and is capable of displaying the secondimage generated by the information processing apparatus. For example,the other display device may be integrated with (in one housing) theinformation processing apparatus.

According to the above configuration, an image of the same virtualworld, which is different from the image displayed on the portabledisplay device, is displayed on the other display device connected tothe information processing device. Therefore, images of the virtualworld, which are formed in accordance with different purposes of use,can be displayed on the display devices favorable for the user,respectively.

The display control section may display, on the other display device, asthe second image, an image of the virtual world including the firstobject and the second object, in which the virtual world is viewed froma viewpoint different from the viewpoint of the virtual world of thefirst image.

According to the above configuration, an image of the virtual world,which is viewed from a viewpoint different from that of the imagedisplayed on the portable display device, is displayed on the otherdisplay device connected to the information processing device.Therefore, images formed in accordance with different purposes of usecan be displayed on the display devices favorable for the user,respectively.

The display control section may set a viewpoint of the virtual world ofthe second image, in a position apart from the first object and thesecond object by a distance longer than the distance from the viewpointof the virtual world of the first image to the first object and thesecond object. The display control section may display, on the otherdisplay device, as the second image, a range of the virtual world, whichis larger than the range of the virtual world shown by the first image.

According to the above configuration, an image of the virtual world,which is larger in display range than the image of the virtual worlddisplayed on the portable display device, is displayed on the otherdisplay device connected to the information processing device.Therefore, when providing the user with the state of the virtual world,images suitable for the user's operation can be displayed on therespective display devices.

The display control section may set a viewpoint for generating thesecond image, in a position in the virtual world from which the firstobject and the second object are viewed in a manner of bird's eyeviewing. The display control section may display, on the other displaydevice, as the second image, an image in which the first object and thesecond object arranged in the virtual world are viewed in a manner ofbird's eye viewing.

According to the above configuration, an image in which the virtualworld is viewed in a manner of bird's eye viewing is displayed on theother display device connected to the information processing device.Therefore, when providing the user with the state of the virtual world,images suitable for the user's operation can be displayed on therespective display devices.

The display control section may set a first virtual camera forgenerating the first image and a second virtual camera for generatingthe second image, and cause a line-of-sight direction of the secondvirtual camera in the virtual world to change in accordance with achange in a line-of-sight direction of the first virtual camera in thevirtual world.

According to the above configuration, it is possible to realize adisplay control such that when the direction of the virtual worlddisplayed on the portable display device changes, the direction of thevirtual world displayed on the other display device also changes in thesame direction.

The display control section may arrange the first virtual camera suchthat a forward direction of the second object from behind the secondobject in the virtual world is an imaging direction of the first virtualcamera, and arrange the second virtual camera such that a position inthe forward direction of the second object in the virtual world is apoint of view of the second virtual camera, and the second object isdisplayed in the second image at a constant position with respect to thepoint of view. The display control section may control the positionsand/or the attitudes of the first virtual camera and the second virtualcamera in accordance with the action of the second object.

According to the above configuration, it is possible to realize adisplay control such that the image of the virtual world displayed onthe portable display device and the image of the virtual world displayedon the other display device change in accordance with only the action ofthe second object.

The attitude calculation section may calculate an attitude of theportable display device with respect to the a predetermined direction inthe real space. The direction determination section may determinewhether or not the direction of the predetermined axis is within thefirst range which is set with respect to the predetermined direction inthe real space, based on the attitude of the portable display devicewith respect to the predetermined direction in the real space. When theresult of the determination is that the direction of the predeterminedaxis with respect to the predetermined direction in the real space iswithin the first range, the first object action control section maycontrol the action of the first object, with reference to apredetermined direction which is set in the virtual world andcorresponds to the predetermined direction in the real space, based onthe direction of the predetermined axis with respect to thepredetermined direction in the real space. When the result of thedetermination is that the direction of the predetermined axis withrespect to the predetermined direction in the real space is outside thefirst range, the second object action control section may control theaction of the second object with reference to the predetermineddirection set in the virtual world, based on the direction of thepredetermined axis with respect to the predetermined direction in thereal space.

According to the above configuration, it is possible to set an action ofthe object with respect to the direction in the virtual world, whichcorresponds to the predetermined direction, in accordance with theattitude of the portable display device with respect to thepredetermined direction.

The attitude calculation section may calculate, with a direction ofgravity in the real space being the predetermined direction, an attitudeof the portable display device with respect to the direction of gravity.The direction determination section may set, with the direction ofgravity in the real space being the predetermined direction in the realspace, the first range with respect to the direction of gravity, anddetermine whether or not the direction of the predetermined axis iswithin the first range, based on the attitude of the portable displaydevice with respect to the direction of gravity in the real space. Whenthe result of the determination is that the direction of thepredetermined axis is within the first range, the first object actioncontrol section may control the action of the first object withreference to the direction of gravity set in the virtual world, based onthe direction of the predetermined axis with respect to the direction ofgravity in the real space. When the result of the determination is thatthe direction of the predetermined axis is outside the first range, thesecond object action control section may control the action of thesecond object with reference to the direction of gravity set in thevirtual world, based on the direction of the predetermined axis withrespect to the direction of gravity in the real space.

According to the above configuration, it is possible to set, based onthe attitude of the portable display device with respect to thedirection of gravity, an action of the object with respect to the samedirection of gravity.

The attitude calculation section may calculate at least an attitude inwhich the portable display device rotates around the direction ofgravity in the real space. The direction determination section may setthe first range in a horizontal direction with respect to the directionof gravity in the real space, and determines whether or not thedirection of the predetermined axis which is projected onto a horizontalplane in the real space is within the first range. When the result ofthe determination is that the direction of the predetermined axisprojected onto the horizontal plane in the real space is within thefirst range, the first object action control section may control theaction of the first object by causing the first object to rotate and/ormove along the horizontal direction set in the virtual world, based onthe direction of the predetermined axis projected onto the horizontalplane. When the result of the determination is that the direction of thepredetermined axis projected onto the horizontal plane in the real spaceis outside the first range, the second object action control section maycontrol the action of the second object by causing the second object torotate and/or move along the horizontal direction set in the virtualworld, based on the direction of the predetermined axis projected ontothe horizontal direction.

According to the above configuration, an operation of rotating and/ormoving the first object in the horizontal direction set in the virtualworld and an operation of rotating and/or moving the second object inthe horizontal direction are realized by turning the portable displaydevice to the left or to the right in the real space.

The attitude calculation section may further calculate an attitude inwhich the portable display device swings up and down around a horizontaldirection perpendicular to the direction of gravity in the real space.The first object action control section may control the action of thefirst object by causing the first object to rotate and/or move around ahorizontal direction which is set in the virtual world and correspondsto the horizontal direction in the real space, based on the attitude inwhich the portable display device swings up and down around thehorizontal direction in the real space, regardless of whether the resultof the determination by the direction determination section is positiveor negative.

According to the above configuration, an operation of rotating and/ormoving the first object around the horizontal direction set in thevirtual world is realized by turning the portable display device to theleft or to the right in the real space.

The attitude calculation section may calculate at least an attitude inwhich the portable display device rotates around two axes orthogonal tothe predetermined axis, the predetermined axis being a depth directionof a display screen of the portable display device, the depth directionbeing perpendicular to the display screen on which the first image isdisplayed. The direction determination section may set the first rangearound one of the two axes, and determine whether or not the depthdirection rotated around the one axis is within the first range. Whenthe result of the determination is that the depth direction is withinthe first range, the first object action control section may cause thefirst object to rotate and/or move around an axis of a verticaldirection in the virtual world, based on the depth direction rotatedaround the one axis. When the result of the determination is that thedepth direction is outside the first range, the second object actioncontrol section may cause the second object to rotate or move around theaxis of the vertical direction in the virtual world, based on the depthdirection which has rotated around the one axis. The first object actioncontrol section may cause the first object to rotate and/or move aroundthe horizontal direction in the virtual world, based on the depthdirection rotated around the other axis, regardless of whether theresult of the determination by the direction determination section ispositive or negative.

According to the above configuration, an operation of rotating and/ormoving the first object around the axis of the vertical direction set inthe virtual world and an operation of rotating and/or moving the secondobject around the axis of the vertical direction are realized by causingthe portable display device to rotate around one of the two axesorthogonal to the depth direction of the display screen of the portabledisplay device in the real space. Further, an operation of rotatingand/or moving the first object around the axis of the horizontaldirection set in the virtual world is realized by causing the portabledisplay device to rotate the other of the two axes orthogonal to thedepth direction of the display screen of the portable display device inthe real space.

The attitude calculation section may calculate at least an attitude inwhich the portable display device rotates around an axis along ahorizontal direction of a display screen of the portable display deviceand an axis along a vertical direction of the display screen, the axesbeing orthogonal to the depth direction. The direction determinationsection may set the first range around the axis along the verticaldirection of the display screen, and determine whether or not the depthdirection rotated around the axis along the vertical direction is withinthe first range. When the result of the determination is that the depthdirection is within the first range, the first object action controlsection may cause the first object to rotate and/or move around the axisof the vertical direction in the virtual world, based on the depthdirection rotated around the axis of the vertical direction of thedisplay screen. When the result of the determination is that the depthdirection is outside the first range, the second object action controlsection may cause the second object to rotate and/or move around theaxis of the vertical direction in the virtual world, based on the depthdirection rotated around the axis of the vertical direction of thedisplay screen. The first object action control section may cause thefirst object to rotate and/or move around the horizontal direction inthe virtual world, based on the depth direction rotated around the axisof the horizontal direction of the display screen, regardless of whetherthe result of the determination by the direction determination sectionis positive or negative.

According to the above configuration, an operation of rotating and/ormoving the first object around the axis of the vertical direction set inthe virtual world and an operation of rotating and/or moving the secondobject around the vertical direction are realized by causing theportable display device to rotate around the axis along the verticaldirection of the display screen of the portable display device in thereal space.

the information processing program may further cause the computer to actas a correction section. The correction section corrects the directionof the predetermined axis set in the portable display device, by apredetermined angle, in a direction along which a display screen of theportable display device, which faces upward, stands up in a verticaldirection. In this case, the direction determination section maydetermine whether or not the direction of the predetermined axis, whichis corrected by the correction section, is within the first range. Thefirst object action control section may control the action of the firstobject in accordance with the direction of the predetermined axis whichis corrected by the correction section. The second object action controlsection may control the action of the second object in accordance withthe direction of the predetermined axis which is corrected by thecorrection section.

According to the above configuration, the attitude of the portabledisplay device is corrected from the attitude in which the displayscreen faces upward to the attitude in which the display screen standsup in the vertical direction. Therefore, the user is allowed to operatethe portable display device with the display screen of the portabledisplay device facing upward so that he/she can easily view the displayscreen.

The first object action control section may cause, as the first object,at least a target object which is a target of an attack according to anoperation by a user, to move in the virtual world in accordance with thepredetermined axis, when the result of the determination is that thedirection of the predetermined axis is within the first range. Thedisplay control section may include a display range setting sectionconfigured to set a display range of the virtual world to be displayedon the portable display device, based on the action of the secondobject.

According to the above configuration, the target is moved by changingthe attitude of the portable display device such that the direction ofthe predetermined axis set in the portable display device is within thefirst range, and the display range of the virtual world displayed on theportable display device is shifted by changing the attitude of theportable display device such that the direction of the predeterminedaxis is outside the first range. Accordingly, the user is allowed toselectively use the operation of moving the target and the operation ofshifting the display range, depending on the manner of changing theattitude of the portable display device.

The display range setting section may set the display range based on theaction of the second object, by arranging a virtual camera forgenerating an image of the virtual world such that the first object andthe second object are included in the display range, with a forwarddirection of the second object from behind the second object in thevirtual world being a line-of-sight direction of the virtual camera. Thedisplay control section may display, on the portable display device, asthe first image, an image showing the virtual world viewed from thevirtual camera.

According to the above configuration, since the virtual camera islocated behind the second object, a realistic image can be displayed onthe portable display device.

When the result of the determination is that the direction of thepredetermined axis is within the first range, the first object actioncontrol section may cause, as the first object, a shooting object whichshoots a predetermined object toward the target to rotate in the virtualworld in accordance with the direction of the predetermined axis. Whenthe result of the determination is that the direction of thepredetermined axis is outside the first range, the second object actioncontrol section may cause the second object to rotate integrally withthe first object in the virtual world in accordance with the directionof the predetermined axis, and cause the target object to move in thevirtual world in accordance with the direction of the shooting object.

According to the above configuration, the direction of the shootingobject is changed and the target of the predetermined object to be shotby the shooting object is moved by changing the attitude of the portabledisplay device such that the direction of the predetermined axis set inthe portable display device is within the first range. Further, thedirection of the second object is changed integrally with the shootingobject and the display range of the virtual world displayed on theportable display device is shifted by changing the attitude of theportable display device such that the direction of the predeterminedaxis is outside the first range.

The portable display device may include at least one of a gyro sensorand an acceleration sensor. In this case, the attitude calculationsection may calculate an attitude of the portable display device, basedon data outputted from at least one of the gyro sensor and theacceleration sensor.

According to the above configuration, it is possible to accuratelycalculate an attitude of the portable display device by using data whichis output from the gyro sensor and represents an angular velocity thatoccurs in the portable display device and/or data which is output fromthe acceleration sensor and represents an acceleration that occurs inthe portable display device.

The information processing program may further cause the computer to actas a load obtaining section and an object moving section. The loadobtaining section obtains data based on a load applied to the loaddetection device. The object moving section causes the first object andthe second object to move in the direction of the second object in thevirtual world, based on the data obtained by the load obtaining section.

The load detection device is, for example, a device which detects a loadapplied thereto when a user puts at least a part of his/her bodythereon, or a device which outputs an on/off signal according to theload. It is considered that, when the user operates the load detectiondevice, the user may stand on the load detection device with his/herbottoms of both feet contacting the upper surface thereof, or the usermay put his/her one foot on the upper surface thereof, or the user mayput another part (e.g., a hand) of his/her body on the upper surfacethereof.

According to the above configuration, it is possible to realize aprocess of moving the first object and the second object based on anoperation performed by the user using the load detection device, and theuser is allowed to view the result of the process on the portabledisplay device. Therefore, the user is allowed to view an imagedisplayed on the portable display device while holding the portabledisplay device. Accordingly, the user is allowed to perform an operationin a new operation environment, and allowed to view the result of theprocess in a situation suited to the operation environment and favorablefor the user. Further, since the movement direction of the first objectand the second object is the direction of the second object which is setbased on the attitude of the portable display device, the first objectand the second object can be moved in accordance with the direction towhich the portable display device is directed, and therefore, the useris allowed to operate the first object and the second object by movingthe portable display device.

The information processing program may further causes the computer toact as a center-of-gravity position calculation section. Thecenter-of-gravity position calculation section calculates acenter-of-gravity position of the load applied to the load detectiondevice, based on the data obtained by the load obtaining section. Inthis case, the object moving section may cause the first object and thesecond object to move in the direction of the second object, based onthe center-of-gravity position calculated by the center-of-gravityposition calculation section.

According to the above configuration, the user is allowed to move thefirst object and the second object along the set movement direction, inaccordance with the center-of-gravity position of the load applied tothe load detection device by the user. For example, the user is allowedto move the first object and the second object along the set movementdirection by performing stepping or shifting the center-of-gravityposition of his/her body forward, backward, leftward, or rightward onthe load detection device.

The object moving section may set a movement speed of the first objectand the second object, based on a distance from a preset referenceposition to the center-of-gravity position calculated by thecenter-of-gravity position calculation section, set a forward movementor a backward movement of the first object and the second object, basedon the distance from the reference position to the center-of-gravityposition, and cause the first object and the second object to move inthe direction of the second object.

The object moving section may set, as the reference position, a straightline which divides a part of the load detection device to which the loadis applied, into a front part and a rear part, set a movement speed ofthe first object and the second object, based on a distance from thestraight line to the center-of-gravity position calculated by thecenter-of-gravity position calculation section, set the first object andthe second object to a forward movement when the center-of-gravityposition is located in the front part with respect to the straight line,and set the first object and the second object to a backward movementwhen the center-of-gravity position is located in the rear part withrespect to the straight line.

According to the above configuration, the user is allowed to move thefirst object and the second object forward or backward in the directionalong the direction of the second object by shifting thecenter-of-gravity position of the load applied to the load detectiondevice by the user. For example, when the user tilts his/her body so asto put his/her weight forward on the load detection device, the firstobject and the second object are moved forward along the direction ofthe second object at a speed according to the amount of the forwardshift of the center-of-gravity position. When the user tilts his/herbody so as to put his/her weight backward on the load detection device,the first object and the second object are moved backward along thedirection opposite to the direction of the second object at a speedaccording to the amount of the backward shift of the center-of-gravityposition.

The display control section may output, to the portable display device,image data representing the first image. The portable display device mayinclude an image data obtaining section and a display section. The imagedata obtaining section obtains image data outputted from the informationprocessing apparatus. The display section displays the first imagerepresented by the image data obtained by the image data obtainingsection.

According to the above configuration, the portable display devicefunctions as a so-called thin client terminal which does not executeinformation processing.

The information processing program may further cause the computer to actas a compressed image generation section. The compressed imagegeneration section generates compressed image data by compressing theimage data representing the first image. In this case, the displaycontrol section may output the compressed image data generated by thecompressed image generation section to the portable display device. Theimage data obtaining section may obtain the compressed image dataoutputted from the information processing apparatus. The portabledisplay device may further include a display image decompression sectionconfigured to decompress the compressed image data to obtain image datarepresenting the first display image. The display section may displaythe first image represented by the image data which has been obtained bythe image data obtaining section and decompressed by the display imagedecompression section.

According to the above configuration, since the first image iscompressed and transmitted from the information processing apparatus tothe portable display device. Therefore, the first image can betransmitted at a high speed, thereby reducing a delay from when thefirst image is generated to when the first image is displayed on theportable display device.

The exemplary embodiments may be implemented in the form of aninformation processing system including the above-mentioned section, orin the form of an information processing method including actionsperformed by the above-mentioned section.

According to the exemplary embodiments, a user is allowed to easilyperform an operation when he/she operates a plurality of objects in avirtual world.

These and other objects, features, aspects and advantages of theexemplary embodiments will become more apparent from the followingdetailed description when taken in conjunction with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an external view illustrating a non-limiting example of a gamesystem 1 according to an embodiment;

FIG. 2 is a functional block diagram illustrating a non-limiting exampleof a game apparatus body 5 shown in FIG. 1;

FIG. 3 is a diagram illustrating a non-limiting example of an externalstructure of a terminal device 6 shown in FIG. 1;

FIG. 4 is a diagram illustrating a non-limiting example of a state wherea user holds the terminal device 6;

FIG. 5 is a block diagram illustrating a non-limiting example of aninternal structure of the terminal device 6 shown in FIG. 3;

FIG. 6 is a perspective view illustrating a non-limiting example of aboard type controller 9 shown in FIG. 1;

FIG. 7 is a diagram illustrating a non-limiting example of across-sectional view of the board type controller 9 taken along a lineA-A in FIG. 6, and a non-limiting example of an enlarged view of acorner portion in which a load sensor 94 is provided;

FIG. 8 is a block diagram illustrating a non-limiting example of anelectric configuration of the board type controller 9 shown in FIG. 6;

FIG. 9 is a diagram illustrating a non-limiting example of a user whoperforms an operation by using the terminal device 6 and the board typecontroller 9;

FIG. 10A is a diagram illustrating a non-limiting example of an imagedisplayed on a LCD 61 of the terminal device 6 in a first exemplarygame;

FIG. 10B is a diagram illustrating a non-limiting example of an imagedisplayed on a monitor 2 in the first exemplary game;

FIG. 11 is a diagram illustrating a non-limiting example of main dataand programs stored in a main memory of the game apparatus body 5 shownin FIG. 1 in the first exemplary game;

FIG. 12 is a flowchart illustrating a non-limiting example of a processexecuted by the game apparatus body 5 shown in FIG. 1;

FIG. 13 is a sub-routine illustrating a non-limiting example of a gamecontrol process in step 44 in FIG. 12;

FIG. 14 is a sub-routine illustrating a non-limiting example of a playerobject setting process in step 83 in FIG. 13;

FIG. 15 is a sub-routine illustrating a non-limiting example of a playerobject action setting process in step 126 in FIG. 14;

FIG. 16 is a diagram illustrating a non-limiting example of arelationship between a real space reference direction and a currentdirection which are projected on a horizontal plane in a real space, anda virtual world reference direction and an operation instructiondirection which are projected on a horizontal plane in a virtual world;

FIG. 17 is a diagram illustrating a non-limiting example of arelationship between a current direction projected on a vertical planein the real space, and an operation instruction direction projected on avertical plane in the virtual world;

FIG. 18A is a diagram illustrating a non-limiting example of an imagedisplayed on the LCD 61 of the terminal device 6 in a second exemplarygame;

FIG. 18B is a diagram illustrating a non-limiting example of an imagedisplayed on the monitor 2 in the second exemplary game;

FIG. 19 is a diagram illustrating an another non-limiting example of animage displayed on the LCD 61 of the terminal device 6 in the secondexemplary game;

FIG. 20 is a schematic diagram illustrating a non-limiting example of astate in which a position and an attitude of a virtual camera changes inaccordance with the attitude of the terminal device 6;

FIG. 21 is a diagram illustrating a non-limiting example of main dataand programs stored in the main memory of the game apparatus body 5 inthe second exemplary game;

FIG. 22 is a sub-routine illustrating a non-limiting example of a playerobject setting process according to a second embodiment, which isexecuted in the game apparatus body 5;

FIG. 23 is a sub-routine illustrating a non-limiting example of ajumping action setting process in step 165 in FIG. 22;

FIG. 24 is a diagram illustrating a non-limiting example of a “squat”determination and a “stretch” determination with respect to variation ina total load value;

FIG. 25A is a diagram illustrating a non-limiting example of an imagedisplayed on the LCD 61 of the terminal device 6 in a third exemplarygame;

FIG. 25B is a diagram illustrating a non-limiting example of an imagedisplayed on the monitor 2 in the third exemplary game;

FIG. 26 is a schematic diagram illustrating a non-limiting example ofactions of a first player object Po1 and a second player object Po2 inaccordance with the attitude of the terminal device 6;

FIG. 27 is a diagram illustrating a non-limiting example of main dataand programs stored in the main memory of the game apparatus body 5 inthe third exemplary game;

FIG. 28 is a sub-routine illustrating a non-limiting example of a playerobject setting process according to a third embodiment, which isexecuted in the game apparatus body 5; and

FIG. 29 is a sub-routine illustrating a non-limiting example of anoperation direction calculation process in step 201 in FIG. 28.

DETAILED DESCRIPTION OF NON-LIMITING EXAMPLE EMBODIMENTS FirstEmbodiment

With reference to FIG. 1, an information processing apparatus forexecuting an information processing program according to a firstembodiment and an information processing system including theinformation processing apparatus will be described. Hereinafter, inorder to provide a specific description, a stationary game apparatusbody 5 is used as an example of the information processing apparatus,and a game system including the game apparatus body 5 will be described.FIG. 1 is an external view illustrating an example of the game system 1including the stationary game apparatus body 5. FIG. 2 is a blockdiagram illustrating an example of the game apparatus body 5.Hereinafter, the game system 1 will be described.

As shown in FIG. 1, the game system 1 includes a household televisionreceiver (hereinafter, referred to as a monitor) 2 which is an exampleof display means, and the stationary game apparatus 3 connected to themonitor 2 via a connection cord. The monitor 2 includes loudspeakers 2 afor outputting, in the form of sound, a sound signal outputted from thegame apparatus 3. Further, the game apparatus 3 includes: an opticaldisc 4 having stored therein a program (e.g., a game program); the gameapparatus body 5 having a computer for executing the program stored inthe optical disc 4 to display a game screen on the monitor 2; a terminaldevice 6; a controller 7 for providing the game apparatus body 5 withoperation information required for operating, for example, objectsdisplayed on the display screen; and a board type controller 9. The gamesystem 1 executes a game process on the game apparatus body 5 inaccordance with a game operation using at least one of the terminaldevice 6, the controller 7, and the board type controller 9, anddisplays a game image obtained by the game process on the monitor 2and/or the terminal device 6. The game apparatus body 5 is wirelesslyconnected to the terminal device 6, the controller 7, and the board typecontroller 9 so as to enable wireless communication therebetween. Forexample, the wireless communication is executed according to theBluetooth (registered trademark) standard or the IEEE802.11n standard.However, the wireless communication may be executed in accordance withother standards such as standards for infrared communication.

The optical disc 4, typifying an information storage medium used for thegame apparatus body 5 in an exchangeable manner, is detachably insertedin the game apparatus body 5. The optical disc 4 has stored therein aninformation processing program (typically, a game program) to beexecuted by the game apparatus body 5. The game apparatus body 5 has, ona front surface thereof, an insertion opening for the optical disc 4.The game apparatus body 5 reads and executes the information processingprogram stored in the optical disc 4 inserted into the insertion openingto execute the game process.

The monitor 2 is connected to the game apparatus body 5 via a connectioncord. The monitor 2 displays a game image obtained by the game processexecuted by the game apparatus body 5. The monitor 2 includes theloudspeakers 2 a. The loudspeakers 2 a each output a game sound obtainedas a result of the game process. In another embodiment, the gameapparatus body 5 and a stationary display unit may be integrated witheach other. The communication between the game apparatus body 5 and themonitor 2 may be wireless communication.

The game apparatus body 5 has mounted thereto a flash memory 17 (seeFIG. 2) which functions as a backup memory for fixedly storing data suchas saved data. The game apparatus body 5 executes the game program orthe like stored in the optical disc 4, and displays a result thereof asa game image on the monitor 2 and/or the terminal device 6. The gameprogram or the like to be executed may be previously stored in the flashmemory 17 as well as in the optical disc 4. Further, the game apparatusbody 5 may reproduce a state of a game played in the past, by using thesaved data stored in the flash memory 17, and display an image of thegame state on the monitor 2 and/or the terminal device 6. A user of thegame apparatus 3 can enjoy the game progress by operating at least oneof the terminal device 6, the controller 7, and the board typecontroller 9 while viewing the game image displayed on the monitor 2and/or the terminal device 6.

The controller 7 and the board type controller 9 each wirelesslytransmit transmission data such as operation information, by using, forexample, the Bluetooth technology, to the game apparatus body 5 having acontroller communication module 19. The controller 7 is operation meansfor performing, for example, selection of options displayed on thedisplay screen of the monitor 2. The controller 7 includes a housingwhich is small enough to be held by one hand, and a plurality ofoperation buttons (including a cross key and the like) which are exposedat the surface of the housing. In addition, as is described later, thecontroller 7 includes an imaging information calculation section fortaking an image viewed from the controller 7. As exemplary imagingtargets of the imaging information calculation section, two LED modules(hereinafter, referred to as “markers”) 8L and 8R are provided in thevicinity of the display screen of the monitor 2 (above the screen inFIG. 1). Although details will be described later, a user (player) isallowed to perform a game operation while moving the controller 7, andthe game apparatus body 5 uses a marker 8 to calculate the movement,position, attitude and the like of the controller 7. The marker 8 hastwo markers 8L and 8R at both ends thereof. Specifically, the marker 8L(as well as the marker 8R) includes one or more infrared LEDs (LightEmitting Diodes), and emits infrared light forward from the monitor 2.The marker 8 is connected to the game apparatus body 5, so that the gameapparatus body 5 can control the infrared LEDs included in the marker 8to be lit on or off. The marker 8 is a portable unit, so that the useris allowed to place the marker 8 in any position. Although FIG. 1 showsa case where the marker 8 is placed on the monitor 2, the location anddirection of the marker 8 may be optionally selected. Further, thecontroller 7 is capable of receiving, at a communication section 75,transmission data wirelessly transmitted from the controllercommunication module 19 of the game apparatus body 5, to generate asound or vibration based on the transmission data.

In another embodiment, the controller 7 and/or the board type controller9 may be connected to the game apparatus body 5 via a cable. Further, inthe exemplary embodiment, the game system 1 includes a controller 7 anda board type controller 9. However, the game apparatus body 5 iscommunicable with a plurality of controllers 7 and a plurality of boardtype controllers 9. Therefore, a plurality of players can play a game byusing a predetermined number of controllers 7 and board type controller9 simultaneously.

The controller 7 includes a housing which is formed by, for example,plastic molding, and has a plurality of operation sections in thehousing. The controller 7 transmits, to the game apparatus body 5,operation data representing an input state to any of the operationsections (i.e., whether or not any of the operation buttons is pressed).

The controller 7 includes an imaging information calculation section foranalyzing image data of an image taken by image pickup means to identifyan area having a high luminance in the image, and calculates theposition of the center of gravity, the size, and the like of the area.For example, the imaging information calculation section includes imagepickup means fixed to the housing of the controller 7. A marker emittinginfrared light, such as a marker section 65 of the terminal device 6and/or the marker 8, is a target to be imaged by the image pickup means.The imaging information calculation section calculates the position ofthe imaging target in the image taken by the image pickup means, andtransmits, to the game apparatus body 5, marker coordinate datarepresenting the calculated position. Since the marker coordinate datavaries depending on the direction (angle of tilt) or the position of thecontroller 7, the game apparatus body 5 can calculate the direction andthe position of the controller 7 by using the marker coordinate data.

The controller 7 includes an acceleration sensor and/or a gyro sensor.The acceleration sensor detects an acceleration (including gravitationalacceleration) of the controller 7, and transmits acceleration datarepresenting the detected acceleration to the game apparatus body 5.Since the acceleration detected by the acceleration sensor variesdepending on the direction (angle of tilt) and the movement of thecontroller 7, the game apparatus body 5 can calculate the direction andthe movement of the controller 7 by using the obtained accelerationdata. The gyro sensor detects angular velocities around three axes whichare set in the controller 7, and transmits angular velocity datarepresenting the detected angular velocities to the game apparatus body5. Since the angular velocities detected by the gyro sensor varydepending on the direction (angle of tilt) and the movement of thecontroller 7, the game apparatus body 5 can calculate the direction andthe movement of the controller 7 by using the obtained angular velocitydata. In this way, the user is allowed to perform a game operation bypressing the operation sections provided on the controller 7, or bymoving the controller 7 to change the position and the attitude (angleof tilt) of the controller 7.

The controller 7 is provided with a loudspeaker and a vibrator. Thecontroller 7 processes sound data transmitted from the game apparatusbody 5, and causes the loudspeaker to output a sound based on the sounddata. Further, the controller 7 processes vibration data transmittedfrom the game apparatus body 5, and causes the vibrator to vibrate inaccordance with the vibration data. In the exemplary embodimentdescribed later, a user is allowed to play a game without using thecontroller 7. The board type controller 9 will be described in detaillater.

The terminal device 6 is a portable device that is small enough to beheld by a user, and the user is allowed to move the terminal device 6with hands, or place the terminal device 6 at any location. Although aspecific structure of the terminal device 6 will be described later, theterminal device 6 includes an LCD (Liquid Crystal Display) 61 as displaymeans, and input means (a touch panel 62, a gyro sensor 604, and thelike described later). The terminal device 6 and the game apparatus body5 (a terminal communication module 28 (see FIG. 2)) are communicablewith each other wirelessly or via a cable. The terminal device 6receives, from the game apparatus body 5, data of an image (e.g., a gameimage) generated in the game apparatus body 5, and displays the imagerepresented by the data on an LCD 61. Although in the exemplaryembodiment the LCD 61 is used as a display device, the terminal device 6may include any other display device, such as a display device utilizingEL (Electro Luminescence), for example. Further, the terminal device 6transmits, to the game apparatus body 5 having the terminalcommunication module 28, operation data representing the content of anoperation performed on the terminal device 6.

Next, with reference to FIG. 2, an internal structure of the gameapparatus body 5 will be described. FIG. 2 is a block diagramillustrating an example of an internal structure of the game apparatusbody 5. The game apparatus body 5 includes a CPU (Central ProcessingUnit) 10, a system LSI (Large Scale Integration) 11, an external mainmemory 12, a ROM/RTC (Read Only Memory/Real Time Clock) 13, a disc drive14, an AV-IC (Audio Video-Integrated Circuit) 15 and the like.

The CPU 10, serving as a game processor, executes a program stored inthe optical disc 4 to perform a process. The CPU 10 is connected to thesystem LSI 11. In addition to the CPU 10, the external main memory 12,the ROM/RTC 13, the disc drive 14, and the AV-IC 15 are connected to thesystem LSI 11. The system LSI 11 performs processes such as control ofdata transmission between the respective components connected thereto,generation of an image to be displayed, and acquisition of data from anexternal apparatus. An internal structure of the system LSI 11 will bedescribed later. The external main memory 12, which is a volatilememory, stores programs loaded from the optical disc 4 or the flashmemory 17, and stores various data. The external main memory 12 is usedas a work area and a buffer area for the CPU 10. The ROM/RTC 13 includesa ROM (so-called boot ROM) incorporating a program for booting the gameapparatus body 5, and a clock circuit (RTC) for counting time. The discdrive 14 reads, from the optical disc 4, program data, texture data andthe like, and writes the read data into an internal main memory 35described below or the external main memory 12.

The system LSI 11 includes an input/output processor (I/O processor) 31,a GPU (Graphics Processor Unit) 32, a DSP (Digital Signal Processor) 33,a VRAM (Video RAM) 34, and the internal main memory 35. These components31 to 35 are connected to each other via an internal bus (not shown).

The GPU 32, which is a part of rendering means, generates an image inaccordance with a graphics command (draw command) supplied from the CPU10. The VRAM 34 stores data (such as polygon data and texture data)required for the GPU 32 to execute the graphics command. When an imageis generated, the GPU 32 generates image data by using the data storedin the VRAM 3. In the exemplary embodiment, the game apparatus body 5may generate both a game image to be displayed on the monitor 2 and agame image to be displayed on the terminal device 6. Hereinafter, thegame image to be displayed on the monitor 2 may be referred to as a“monitor game image”, and the game image to be displayed on the terminaldevice 6 may be referred to as a “terminal game image”.

The DSP 33, serving as an audio processor, generates sound data by usingsound data and sound waveform (tone quality) data stored in the internalmain memory 35 and the external main memory 12. In the exemplaryembodiment, similarly to the game images, both a game sound to be outputfrom the loudspeakers 2 a of the monitor 2 and a game sound to be outputfrom the loudspeakers of the terminal device 6 may be generated.Hereinafter, the game sound to be output from the monitor 2 may bereferred to as a “monitor game sound”, and the game sound to be outputfrom the terminal device 6 may be referred to as a “terminal gamesound”.

Among the image data and sound data generated by the game apparatus body5, the image data and sound data to be output to the monitor 2 are readby the AV-IC 15. The AV-IC 15 outputs the read image data to the monitor2 via an AV connector 16, and outputs the read sound data to theloudspeakers 2 a included in the monitor 2. Thereby, an image isdisplayed on the monitor 2, and a sound is output from the loudspeakers2 a.

Further, among the image data and sound data generated by the gameapparatus body 5, the image data and sound data to be output to theterminal device 6 are transmitted to the terminal device 6 by the I/Oprocessor 31 or the like. Data transmission to the terminal device 6 bythe I/O processor 31 or the like will be described later.

The I/O processor 31 executes data reception and transmission with thecomponents connected thereto, and download of data from an externalapparatus. The I/O processor 31 is connected to the flash memory 17, thenetwork communication module 18, the controller communication module 19,an extension connector 20, a memory card connector 21, and a codec LSI27. An antenna 23 is connected to the controller communication module19. The codec LSI 27 is connected to the terminal communication module28, and an antenna 29 is connected to the terminal communication module28.

The game apparatus body 5 is connected to a network such as the Internetso as to communicate with external information processing apparatuses(for example, other game apparatuses or various servers). That is, theI/O processor 31 is connected to a network via the network communicationmodule 18 and the antenna 22 so as to communicate with externalinformation processing apparatuses connected to the network. The I/Oprocessor 31 accesses the flash memory 17 at regular intervals so as todetect for data to be transmitted to the network. When data to betransmitted is detected, the data is transmitted to the network via thenetwork communication module 18 and the antenna 22. Further, the I/Oprocessor 31 receives, via the network, the antenna 22 and the networkcommunication module 18, data transmitted from the external informationprocessing apparatuses or data downloaded from a download server, andstores the received data in the flash memory 17. The CPU 10 executes aprogram, and reads the data stored in the flash memory 17 to use thedata for execution of the program. The flash memory 17 may store notonly the data transmitted and received between the game apparatus body 5and the external information processing apparatuses, but also saved data(result data or progress data of the process) of the game played withthe game apparatus body 5. Further, the flash memory 17 may storeprograms such as a game program.

The game apparatus body 5 can receive operation data from the controller7 and/or the board type controller 9. That is, the I/O processor 31receives, via the antenna 23 and the controller communication module 19,operation data or the like transmitted from the controller 7 and/or theboard type controller 9, and stores (temporarily) the data in a bufferregion of the internal main memory 35 or the external main memory 12.Similarly to the external main memory 12, the internal main memory 35may store a program loaded from the optical disc 4 or a program loadedfrom the flash memory 17, and various data. The internal main memory 35may be used as a work region or buffer region of the CPU 10.

The game apparatus body 5 is capable of transmitting/receiving imagedata, sound data and the like to/from the terminal device 6. Whentransmitting a game image (terminal game image) to the terminal device6, the I/O processor 31 outputs data of a game image generated by theGPU 32 to the codec LSI 27. The codec LSI 27 performs a predeterminedcompression process on the image data supplied from the I/O processor31. The terminal communication module 28 performs wireless communicationwith the terminal device 6. Accordingly, the image data compressed bythe codec LSI 27 is transmitted by the terminal communication module 28to the terminal device 6 via the antenna 29. In the exemplaryembodiment, the codec LSI 27 compresses the image data by using a highlyefficient compression technique, for example, the H.264 standard. Thecodec LSI 27 may adopt other compression techniques. When thecommunication rate is sufficiently high, uncompressed image data may betransmitted. The terminal communication module 28 is, for example, aWi-Fi certified communication module. The terminal communication module28 may perform wireless communication with the terminal device 6 at ahigh speed by using, for example, the technique of MIMO (Multiple InputMultiple Output) adopted in the IEEE802.11n standard, or may use othercommunication techniques.

The game apparatus body 5 transmits, to the terminal device 6, sounddata as well as the image data. That is, the I/O processor 31 outputssound data generated by the DSP 33 to the terminal communication module28 via the codec LSI 27. The codec LSI 27 performs a compression processon the sound data in a similar manner to that for the image data. Anycompression technique may be adopted for the sound data. In anotherembodiment, uncompressed sound data may be transmitted. The terminalcommunication module 28 transmits the compressed image data and sounddata to the terminal device 6 via the antenna 29.

The game apparatus body 5 transmits, in addition to the image data andsound data, various control data to the terminal device 6, according toneed. The control data represent control instructions for the componentsincluded in the terminal device 6, such as an instruction to controlon/off of a marker section (a marker section 65 shown in FIG. 5), and aninstruction to control image taking of a camera (a camera 66 shown inFIG. 5). The I/O processor 31 transmits the control data to the terminaldevice 6 in response to an instruction from the CPU 10. In the exemplaryembodiment, the codec LSI 27 does not perform a data compression processon the control data. In another embodiment, however, the codec LSI 27may perform a compression process on the control data. Theabove-described data transmitted from the game apparatus body 5 to theterminal device 6 may be encrypted according to need, or may not beencrypted.

The game apparatus body 5 can receive various data from the terminaldevice 6. Although details will be described later, in the exemplaryembodiment, the terminal device 6 transmits operation data, image data,and sound data. The respective data transmitted from the terminal device6 are received by the terminal communication module 28 via the antenna29. The image data and sound data transmitted from the terminal device 6have been subjected to a similar compression process to that for theimage data and sound data transmitted from the game apparatus body 5 tothe terminal device 6. Accordingly, these image data and sound data aretransmitted from the terminal communication module 28 to the codec LSI27, and subjected to a decompression process by the codec LSI 27. Thedecompressed data are output to the I/O processor 31. On the other hand,since the operation data transmitted from the terminal device 6 issmaller in amount than the image data and sound data, the operation dataneed not be compressed. The operation data may be encrypted according toneed, or may not be encrypted. Accordingly, the operation data, whichhas been received by the terminal communication module 28, is output tothe I/O processor 31 via the codec LSI 27. The I/O processor 31 stores(temporarily) the data received from the terminal device 6 in the bufferregion of the internal main memory 35 or the external main memory 12.

The game apparatus body 5 is connectable to other devices and externalstorage media. That is, an extension connector 20 and a memory cardconnector 21 are connected to the I/O processor 31. The expansionconnector 20 is an interface connector as typified by a USB and an SCSI,and is capable of performing communication with the network, instead ofthe network communication module 18, by connecting thereto a medium suchas an external storage medium, a peripheral device such as anothercontroller, or a wired communication connector. The memory cardconnector 21 is a connector for connecting thereto an external storagemedium such as a memory card. For example, the I/O processor 31 accessesthe external storage medium via the expansion connector 20 or the memorycard connector 21 to save or read data.

The game apparatus body 5 includes (on the front main surface thereof,for example) a power button 24, a reset button 25, an insertion slot inwhich the optical disc 4 is inserted, an eject button 26 for ejectingthe optical disc 4 from the insertion slot of the game apparatus body 5,and the like. The power button 24 and the reset button 25 are connectedto the system LSI 11. When the power button 24 is turned on, therespective components of the game apparatus body 5 are supplied withpower. When the reset button 25 is pressed, the system LSI 11re-executes the boot program of the game apparatus body 5. The ejectbutton 26 is connected to the disc drive 14. When the eject button 26 ispressed, the optical disc 4 is ejected from the disc drive 14.

In another embodiment, some of the components of the game apparatus body5 may be constituted as an extension device separated from the gameapparatus body 5. At this time, the extension device may be connected tothe game apparatus body 5 via the extension connector 20. Specifically,the extension device may include, for example, the codec LSI 27, theterminal communication module 28, and the antenna 29, and may bedetachably connected to the extension connector 20. Thus, by connectingthe extension device to the game apparatus body which does not have theabove-mentioned, the game apparatus body can be made communicable withthe terminal device 6.

Next, a structure of the terminal device 6 will be described withreference to FIGS. 3 to 5. FIG. 3 is a diagram illustrating an exampleof an external structure of the terminal device 6. More specifically,(a) of FIG. 3 is a front view of the terminal device 6, (b) of FIG. 3 isa top view, (c) of FIG. 3 is a right side view, and (d) of FIG. 3 is abottom view. FIG. 4 shows an example of a state in which a user holdsthe terminal device 6 with both hands.

As shown in FIG. 3, the terminal device 6 includes a housing 60 whichgenerally has a horizontally long plate-like rectangular shape. Thehousing 60 is small enough to be held by the user. Therefore, the useris allowed to move the terminal device 6 with hands, and change thelocation of the terminal device 6.

The terminal device 6 includes an LCD 61 on a front surface of thehousing 60. The LCD 61 is provided near the center of the front surfaceof the housing 60. Therefore, as shown in FIG. 4, the user, holding thehousing 60 at portions to the right and left of the LCD 61, is allowedto move the terminal device 6 while viewing a screen of the LCD 61. FIG.4 shows an example in which the user holds the terminal device 6horizontally (i.e., with the longer sides of the terminal device 6 beingoriented horizontally) by holding the housing 60 at portions to theright and left of the LCD 61. However, the user may hold the terminaldevice 6 vertically (i.e., with the longer sides of the terminal device6 being oriented vertically).

As shown in (a) of FIG. 3, the terminal device 6 includes, as operationmeans, a touch panel 62 on the screen of the LCD 61. In the exemplaryembodiment, the touch panel 62 is, but is not limited to, a resistivefilm type touch panel. However, a touch panel of any type, such aselectrostatic capacitance type, may be used. The touch panel 62 may beof single touch type or multiple touch type. In the exemplaryembodiment, the touch panel 62 has the same resolution (detectionaccuracy) as that of the LCD 61. However, the resolution of the touchpanel 62 and the resolution of the LCD 61 need not be the same. Althoughan input onto the touch panel 62 is usually performed by using a touchpen, in addition to the touch pen, a finger of the user may be used forperforming an input onto the touch panel 62. The housing 60 may have anopening for accommodating the touch pen used for performing an operationto the touch panel 62. Since the terminal device 6 has the touch panel62, the user is allowed to operate the touch panel 62 while moving theterminal device 6. That is, the user is allowed to directly (by usingthe touch panel 62) perform an input onto the screen of the LCD 61 whilemoving the LCD 61.

As shown in FIG. 3, the terminal device 6 has, as operation means, twoanalog sticks 63A and 63B, and a plurality of operation buttons 64A to64L. The analog sticks 63A and 63B are each a device for designating adirection. The analog sticks 63A and 63B are each configured such that astick part thereof to be operated by a finger of the user is slidable ortiltable in any direction (at any angle in any direction such as theupward, the downward, the rightward, the leftward, or the diagonaldirection) with respect to the front surface of the housing 60. The leftanalog stick 63A is provided to the left of the screen of the LCD 61,and the right analog stick 63B is provided to the right of the screen ofthe LCD 61. Therefore, the user is allowed to perform an input fordesignating a direction by using the analog stick 63A or 63B with eitherthe left or right hand. Further, as shown in FIG. 4, the analog sticks63A and 63B are positioned so as to be operated by the user holding theright and left portions of the terminal device 6. Therefore, the user isallowed to easily operate the analog sticks 63A and 63B when the userholds and moves the terminal device 6.

The operation buttons 64A to 64L are each operation means for performinga predetermined input. As described below, the operation buttons 64A to64L are positioned so as to be operated by the user holding the rightand left portions of the terminal device 6 (see FIG. 4). Accordingly,the user is allowed to easily operate the operation means when the userholds and moves the terminal device 6.

As shown in (a) of FIG. 3, among the operation buttons 64A to 64L, thecross button (direction input button) 64A and the operation buttons 64Bto 64H are provided on the front surface of the housing 60. Theoperation buttons 64A to 64H are positioned so as to be operated by athumb of the user (see FIG. 4).

The cross button 64A is provided to the left of the LCD 61 and beneaththe left analog stick 63A. That is, the cross button 64A is positionedso as to be operated by the left hand of the user. The cross button 64Ais cross-shaped, and is capable of indicating an upward, a downward, aleftward, or a rightward direction. The operation buttons 64B to 64D areprovided beneath the LCD 61. The three operation buttons 64B to 64D arepositioned so as to be operated by the right and left hands of the user.The four operation buttons 64E to 64H are provided to the right of theLCD 61 and beneath the right analog stick 63B. That is, the fouroperation buttons 64E to 64H are positioned so as to be operated by theright hand of the user. Further, the four operation buttons 64E, 64H,64F, and 64G are positioned upward, downward, leftward, and rightward,respectively, with respect to a center position of the four operationbuttons. Accordingly, the terminal device 6 may cause the four operationbuttons 64E to 64H to function as buttons which allow the user todesignate an upward, a downward, a leftward, or a rightward direction.

As shown in (a), (b), and (c) of FIG. 3, a first L button 64I and afirst R button 64J are provided on diagonal upper portions (an upperleft portion and an upper right portion) of the housing 60.Specifically, the first L button 64I is provided on the left end of theupper side surface of the plate-shaped housing 60 so as to protrude fromthe upper and left side surfaces. The first R button 64J is provided onthe right end of the upper side surface of the housing 60 so as toprotrude from the upper and right side surfaces. In this way, the firstL button 64I is positioned so as to be operated by the index finger ofthe left hand of the user, and the first R button 64J is positioned soas to be operated by the index finger of the right hand of the user (seeFIG. 4).

As shown in (b) and (c) of FIG. 3, leg parts 68A and 68B are provided soas to protrude from a rear surface (i.e., a surface reverse of the frontsurface on which the LCD 61 is provided) of the plate-shaped housing 60,and a second L button 64K and a second R button 64L are provided so asto protrude from the leg parts 68A and 68B, respectively. Specifically,the second L button 64K is provided at a slightly upper position on theleft side (the left side as viewed from the front surface side) of therear surface of the housing 60, and the second R button 64L is providedat a slightly upper position on the right side (the right side as viewedfrom the front-surface side) of the rear surface of the housing 60. Inother words, the second L button 64K is provided at a positionsubstantially opposite to the left analog stick 63A provided on thefront surface, and the second R button 64L is provided at a positionsubstantially opposite to the right analog stick 63B provided on thefront surface. The second L button 64K is positioned so as to beoperated by the middle finger of the left hand of the user, and thesecond R button 64L is positioned so as to be operated by the middlefinger of the right hand of the user (see FIG. 4). Further, as shown in(c) of FIG. 3, the leg parts 68A and 68B each have a surface facingobliquely upward, and the second L button 64K and the second R button64L are provided on the oblique surfaces of the leg parts 68A and 68B,respectively. Thus, the second L button 64K and the second R button 64Lhave button surfaces facing obliquely upward. Since it is supposed thatthe middle finger of the user moves vertically when the user holds theterminal device 6, the upward facing button surfaces allow the user toeasily press the second L button 64K and the second R button 64L.Further, the leg parts 68A and 68B provided on the rear surface of thehousing 60 allow the user to easily hold the housing 60. Moreover, theoperation buttons provided on the leg parts 68A and 68B allow the userto easily perform operation while holding the housing 60.

In the terminal device 6 shown in FIG. 3, the second L button 64K andthe second R button 64L are provided on the rear surface of the housing60. Therefore, if the terminal device 6 is placed with the screen of theLCD 61 (the front surface of the housing 60) facing upward, the screenof the LCD 61 may not be perfectly horizontal. Accordingly, in anotherembodiment, three or more leg parts may be provided on the rear surfaceof the housing 60. In this case, if the terminal device 6 is placed on afloor with the screen of the LCD 61 facing upward, the three or more legparts contact the floor. Thus, the terminal device 6 can be placed withthe screen of the LCD 61 being horizontal. Such a horizontal placementof the terminal device 6 may be achieved by providing detachable legparts on the rear surface of the housing 60.

The respective operation buttons 64A to 64L are assigned functions,according to need, in accordance with a game program. For example, thecross button 64A may be used for direction designation operation,selection operation, and the like, and the operation buttons 64E to 64Hmay be used for determination operation, cancellation operation, and thelike.

The terminal device 6 includes a power button (not shown) for turningon/off the power of the terminal device 6. The terminal device 6 mayinclude an operation button for turning on/off screen display of the LCD61, an operation button for performing connection setting (pairing) withthe game apparatus body 5, and an operation button for adjusting thevolume of loudspeakers (loudspeakers 607 shown in FIG. 5).

As shown in (a) of FIG. 3, the terminal device 6 includes a markersection (a marker section 65 shown in FIG. 5) including a marker 65A anda marker 65B, on the front surface of the housing 60. For example, themarker section 65 is provided above the LCD 61. The markers 65A and 65Bare each constituted by one or more infrared LEDs, like the markers 8Land 8R of the marker 8. The marker section 65 is used, like the marker8, for causing the game apparatus body 5 to calculate a movement or thelike of the controller 7 with respect to the marker section 65. The gameapparatus body 5 is capable of controlling the infrared LEDs of themarker section 65 to be on or off.

The terminal device 6 includes a camera 66 as imaging means. The camera66 includes an image pickup element (e.g., a CCD image sensor or a CMOSimage sensor) having a predetermined resolution, and a lens. Forexample, the camera 66 is provided on the front surface of the housing60. Accordingly, the camera 66 is capable of taking an image of the faceof the user holding the terminal device 6. For example, the camera 66 iscapable of taking an image of the user playing a game while viewing theLCD 61.

The terminal device 6 has a microphone (a microphone 609 shown in FIG.5) as sound input means. A microphone hole 60 b is provided in the frontsurface of the housing 60. The microphone 609 is embedded in the housing60 at a position inside the microphone hole 60 b. The microphone 609detects for a sound, such as user's voice, around the terminal device 6.

The terminal device 6 has loudspeakers (loudspeakers 607 shown in FIG.5) as sound output means. As shown in (d) of FIG. 3, speaker holes 60 aare provided in the lower side surface of the housing 60. A sound isoutput through the speaker holes 60 a from the loudspeakers 607. In theexemplary embodiment, the terminal device 6 has two loudspeakers, andthe speaker holes 60 a are provided at positions corresponding to a leftloudspeaker and a right loudspeaker, respectively.

The terminal device 6 includes an extension connector 67 for connectinganother device to the terminal device 6. In the exemplary embodiment, asshown in (d) of FIG. 3, the extension connector 67 is provided in thelower side surface of the housing 60. Any device may be connected to theextension connection 67. For example, a controller (a gun-shapedcontroller or the like) used for a specific game or an input device suchas a keyboard may be connected to the extension connector 67. If anotherdevice need not be connected, the extension connector 67 need not beprovided.

In the terminal device 6 shown in FIG. 3, the shapes of the operationbuttons and the housing 60, the number of the respective components, andthe positions in which the components are provided are merely examples.The shapes, numbers, and positions may be different from those describedabove.

Next, an internal structure of the terminal device 6 will be describedwith reference to FIG. 5. FIG. 5 is a block diagram illustrating anexample of an internal structure of the terminal device 6. As shown inFIG. 5, the terminal device 6 includes, in addition to the componentsshown in FIG. 3, a touch panel controller 601, a magnetic sensor 602, agyro sensor 604, a user interface controller (UI controller) 605, acodec LSI 606, loudspeakers 607, a sound IC 608, a microphone 609, awireless module 610, an antenna 611, an infrared communication module612, a flash memory 613, a power supply IC 614, a battery 615, and avibrator 619. These electronic components are mounted on an electroniccircuit board and accommodated in the housing 60.

The UI controller 605 is a circuit for controlling data input to variousinput/output sections and data output from various input/outputsections. The UI controller 605 is connected to the touch panelcontroller 601, the analog stick 63 (the analog sticks 63A and 63B), theoperation button 64 (the operation buttons 64A to 64L), the markersection 65, the magnetic sensor 602, the acceleration sensor 603, thegyro sensor 604, and the vibrator 619. Further, the UI controller 605 isconnected to the codec LSI 606 and the extension connector 67. The powersupply IC 614 is connected to the UI controller 605, so that power issupplied to the respective components through the UI controller 605. Theinternal battery 615 is connected to the power supply IC 614, so thatpower is supplied from the battery 615. Further, a battery charger 616or a cable, which is supplied with power from an external power supply,may be connected to the power supply IC 614 via a connector or the like.In this case, the terminal device 6 can be supplied with power andcharged from the external power supply by using the battery charger 616or the cable. Charging of the terminal device 6 may be performed bysetting the terminal device 6 on a cradle (not shown) having a chargingfunction.

The touch panel controller 601 is a circuit which is connected to thetouch panel 62 and controls the touch panel 62. The touch panelcontroller 601 generates a predetermined form of touch position data,based on a signal from the touch panel 62, and outputs the touchposition data to the UI controller 605. The touch position datarepresents coordinates of a position at which an input is performed onan input surface of the touch panel 62. The touch panel controller 601reads a signal from the touch panel 62 and generates touch position dataevery predetermined period of time. Further, various controlinstructions on the touch panel 62 are output from the UI controller 605to the touch panel controller 601.

The analog stick 63 outputs, to the UI controller 605, stick datarepresenting a direction in which the stick part operated by a finger ofthe user slides (or tilts), and an amount of the sliding (tilting). Theoperation button 64 outputs, to the UI controller 605, operation buttondata representing an input state of each of the operation buttons 64A to64L (whether or not the operation button is pressed).

The magnetic sensor 602 detects the magnitude and direction of amagnetic field to detect an orientation. Orientation data representingthe detected orientation is output to the UI controller 605. The UIcontroller 605 outputs, to the magnetic sensor 602, a controlinstruction for the magnetic sensor 602. Examples of the magnetic sensor602 include: an MI (Magnetic Impedance) sensor, a fluxgate sensor, ahall sensor, a GMR (Giant Magneto Resistance) sensor, a TMR (TunnelingMagneto Resistance) sensor, and an AMR (Anisotropic Magneto Resistance)sensor. However, any sensor may be adopted as long as the sensor candetect an orientation. Strictly speaking, the obtained orientation datadoes not represent an orientation in a place where a magnetic field isgenerated in addition to the geomagnetism. Even in such a case, it ispossible to calculate a change in the attitude of the terminal device 6because the orientation data changes when the terminal device 6 moves.

The acceleration sensor 603 is provided inside the housing 60. Theacceleration sensor 603 detects the magnitudes of linear accelerationsalong three axial directions (x-axis, y-axis, and z-axis directionsshown in (a) of FIG. 3), respectively. Specifically, in the accelerationsensor 603, the long side direction of the housing 60 is defined as thex-axial direction, the short side direction of the housing 60 is definedas the y-axial direction, and the direction orthogonal to the frontsurface of the housing 60 is defined as the z-axial direction, therebydetecting the magnitudes of the linear accelerations in the respectiveaxial directions. Acceleration data representing the detectedaccelerations is output to the UI controller 605. The UI controller 605outputs, to the acceleration sensor 603, a control instruction for theacceleration sensor 603. In the exemplary embodiment, the accelerationsensor 603 is, for example, an electrostatic capacitance type MEMSacceleration sensor. However, in another embodiment, another type ofacceleration sensor may be used. Further, the acceleration sensor 603may be an acceleration sensor for detecting the magnitude ofacceleration in one axial direction or two axial directions.

The gyro sensor 604 is provided inside the housing 60. The gyro sensor604 detects the angular velocities around the three axes (theabove-described x, y, and z axes), respectively. Angular velocity datarepresenting the detected angular velocities is output to the UIcontroller 605. The UI controller 605 outputs, to the gyro sensor 604, acontrol instruction for the gyro sensor 604. Any number and anycombination of gyro sensors may be used as long as the angularvelocities around three axes are detected. The gyro sensor 604 may beconstituted by a two-axis gyro sensor and a one-axis gyro sensor, likethe gyro sensor 703. Alternatively, the gyro sensor 604 may be a gyrosensor for detecting the angular velocity around one axis or two axes.

The vibrator 619 is, for example, a vibration motor or a solenoid. Thevibrator 619 is connected to the UI controller 605. The terminal device6 is vibrated by actuating the vibrator 619 in accordance with a controlinstruction outputted from the UI controller 605 to the vibrator 619.The vibration of the terminal device 6 is transmitted to the user's handholding the terminal device 6. Thus, a so-called vibration-feedback gameis realized.

The UI controller 605 outputs, to the codec LSI 606, the operation dataincluding the touch position data, the stick data, the operation buttondata, the orientation data, the acceleration data, and the angularvelocity data, which have been received from the respective components.If another device is connected to the terminal device 6 through theextension connector 67, data representing operation to the other devicemay be included in the operation data.

The codec LSI 606 is a circuit for performing a compression process ondata to be transmitted to the game apparatus body 5, and a decompressionprocess on data transmitted from the game apparatus body 5. The LCD 61,the camera 66, the sound IC 608, the wireless module 610, the flashmemory 613, and the infrared communication module 612 are connected tothe codec LSI 606. The codec LSI 606 includes a CPU 617 and an internalmemory 618. Although the terminal device 6 is configured not to performa game process, the terminal device 6 may execute a program for managingthe terminal device 6 or a program for communication. For example, aprogram stored in the flash memory 613 is loaded into the internalmemory 618 and executed by the CPU 617 when the terminal device 6 ispowered on, thereby starting up the terminal device 6. A part of thearea of the internal memory 618 is used as a VRAM for the LCD 61.

The camera 66 takes an image in accordance with an instruction from thegame apparatus body 5, and outputs data of the taken image to the codecLSI 606. The codec LSI 606 outputs, to the camera 66, a controlinstruction for the camera 66, such as an instruction to take an image.The camera 66 is also capable of taking a moving picture. That is, thecamera 66 is capable of repeatedly performing image taking, andrepeatedly outputting image data to the codec LSI 606.

The sound IC 608 is connected to the loudspeakers 607 and the microphone609. The sound IC 608 is a circuit for controlling input of sound datafrom the microphone 609 to the codec LSI 606 and output of sound datafrom the codec LSI 606 to the loudspeakers 607. Specifically, when thesound IC 608 receives sound data from the codec LSI 606, the sound IC608 performs D/A conversion on the sound data, and outputs a resultantsound signal to the loudspeakers 607 to cause the loudspeakers 607 tooutput a sound. The microphone 609 detects sound (such as user's voice)propagated to the terminal device 6, and outputs a sound signalrepresenting the sound to the sound IC 608. The sound IC 608 performsA/D conversion on the sound signal from the microphone 609, and outputsa predetermined form of sound data to the codec LSI 606.

The codec LSI 606 transmits the image data from the camera 66, the sounddata from the microphone 609, and the operation data from the UIcontroller 605 (terminal operation data), to the game apparatus body 5through the wireless module 610. In the exemplary embodiment, the codecLSI 606 subjects the image data and the sound data to a compressionprocess similar to that performed by the codec LSI 27. The compressedimage data and sound data, and the terminal operation data are output tothe wireless module 610 as transmission data. The antenna 611 isconnected to the wireless module 610, and the wireless module 610transmits the transmission data to the game apparatus body 5 through theantenna 611. The wireless module 610 has the same function as theterminal communication module 28 of the game apparatus body 5. That is,the wireless module 610 has a function of connecting to a wireless LANby a method based on, for example, the IEEE802.11n standard. The datatransmitted from the wireless module 610 may be encrypted according toneed, or may not be encrypted

As described above, the transmission data transmitted from the terminaldevice 6 to the game apparatus body 5 includes the operation data(terminal operation data), the image data, and the sound data. Ifanother device is connected to the terminal device 6 through theextension connector 67, data received from the other device may beincluded in the transmission data. The infrared communication module 612performs, with another device, infrared communication based on, forexample, the IRDA standard. The codec LSI 606 may include, in thetransmission data, data received by the infrared communication, andtransmit the transmission data to the game apparatus body 5, accordingto need.

As described above, the compressed image data and sound data aretransmitted from the game apparatus body 5 to the terminal device 6.These data are received by the codec LSI 606 through the antenna 611 andthe wireless module 610. The codec LSI 606 decompresses the receivedimage data and sound data. The decompressed image data is output to theLCD 61, and an image according to the image data is displayed on the LCD61. On the other hand, the decompressed sound data is output to thesound IC 608, and a sound based on the sound data is output from theloudspeakers 607.

When control data is included in the data received from the gameapparatus body 5, the codec LSI 606 and the UI controller 605 makecontrol instructions for the respective components, according to thecontrol data. As described above, the control data represents controlinstructions for the respective components (in the exemplary embodiment,the camera 66, the touch panel controller 601, the marker section 65,the sensors 602 to 604, the vibrator 619, and the infrared communicationmodule 612) included in the terminal device 6. In the exemplaryembodiment, the control instructions represented by the control data areconsidered to be instructions to start and halt (stop) the operations ofthe above-mentioned components. That is, some components which are notused for a game may be halted to reduce power consumption. In this case,data from the halted components are not included in the transmissiondata transmitted from the terminal device 6 to the game apparatus body5. Since the marker section 65 is constituted by infrared LEDs, themarker section 65 is controlled by simply turning on/off the supply ofpower thereto.

As described above, the terminal device 6 includes the operation meanssuch as the touch panel 62, the analog sticks 63, and the operationbuttons 64. In another embodiment, however, the terminal device 6 mayinclude other operation means instead of or in addition to theseoperation means.

The terminal device 6 includes the magnetic sensor 602, the accelerationsensor 603, and the gyro sensor 604 as sensors for calculating themovement (including the position and the attitude, or a change in theposition or the attitude) of the terminal device 6. In anotherembodiment, however, the terminal device 6 may include one or two ofthese sensors. In still another embodiment, the terminal device 6 mayinclude other sensors instead of or in addition to these sensors.

The terminal device 6 includes the camera 66 and the microphone 609. Inanother embodiment, however, the terminal device 6 may not include thecamera 66 and the microphone 609, or may include either of the camera 66and the microphone 609.

The terminal device 6 includes the marker section 65 as a component forcalculating the positional relation between the terminal device 6 andthe controller 7 (such as the position and/or the attitude of theterminal device 6 as viewed from the controller 7). In anotherembodiment, however, the terminal device 6 may not include the markersection 65. In still another embodiment, the terminal device 6 mayinclude other means as a component for calculating the above-mentionedpositional relation. For example, the controller 7 may include a markersection, and the terminal device 6 may include an image pickup element.In this case, the marker 8 may include an image pickup element insteadof an infrared LED.

Next, a structure of the board type controller 9 will be described withreference to FIGS. 6 to 8. FIG. 11 is a perspective view illustrating anexample of an external appearance of the board type controller 9 shownin FIG. 1. As shown in FIG. 6, the board type controller 9 includes aplatform 9 a on which a user stands (on which the user puts his/herfeet), and at least four load sensors 94 a to 94 d for detecting a loadapplied to the platform 9 a. Each of the load sensors 94 a to 94 d isembedded in the platform 9 a (see FIG. 7), and the positions where theload sensors 94 a to 94 d are provided are indicated by dotted lines inFIG. 6. In the following description, the four load sensors 94 a to 94 dmay be collectively referred to as a load sensor 94.

The platform 9 a is formed in the shape of substantially a rectangularparallelepiped, and is in the shape of substantially a rectangle asviewed from the top. For example, the short side of the rectangularshape of the platform 9 a is approximately 30 cm, and the long sidethereof is approximately 50 cm. The upper surface of the platform 9 a isflat, and has a pair of planes on which the user stands with the bottomsof his/her feet contacting thereto. Specifically, the upper surface ofthe platform 9 a has a plane (a back-left region enclosed with a doubleline in FIG. 6) on which the user's left foot is put, and a plane (afront-right region enclosed with a double line in FIG. 6) on which theuser's right foot is put. The platform 9 a has, at four corners thereof,side surfaces each partially projecting outward in a cylindrical shape.

In the platform 9 a, the four load sensors 94 a to 94 d are arranged atpredetermined intervals. In the exemplary embodiment, the four loadsensors 94 a to 94 d are arranged on the periphery of the platform 9 a,more specifically, at the four corners of the platform 9 a,respectively. The intervals of the load sensors 94 a to 94 d areappropriately set such that the load sensors 94 a to 94 d can accuratelydetect the intention of a game operation which is expressed by a mannerof applying a load onto the platform 9 a by the user.

FIG. 7 shows an example of a cross-sectional view of the board typecontroller 9, taken along a line A-A in FIG. 6, and an example of anenlarged view of a corner part where a load sensor 94 is arranged. InFIG. 7, the platform 9 a includes a support plate 90 on which the userstands, and legs 92. The load sensors 94 a to 94 d are provided inpositions where the legs 92 are provided. In the exemplary embodiment,since the four legs 92 are provided at the four corners, the four loadsensors 94 a to 94 d are also provided at the corresponding fourcorners. Each leg 92 is formed by plastic molding in the shape ofsubstantially a cylinder with a base. Each load sensor 94 is located ona spherical part 92 a provided on the base of the corresponding leg 92.The support plate 90 is supported by the legs 92 via the load sensors94.

The support plate 90 includes an upper plate 90 a forming an uppersurface and an upper side surface portion, a lower plate 90 b forming alower surface and a lower side surface portion, and an intermediateplate 90 c provided between the upper plate 90 a and the lower plate 90b. The upper plate 90 a and the lower plate 90 b are formed by, forexample, plastic molding, and are integrated using an adhesive or thelike. The intermediate plate 90 c is, for example, formed of a singlemetal plate by press forming. The intermediate plate 90 c is fixed ontothe four load sensors 94 a to 94 d. The upper plate 90 a has, on a lowersurface thereof, a grid-patterned rib (not shown), and is supported bythe intermediate plate 90 c via the rib. Therefore, when the user standson the platform 9 a, the load is transferred to the four legs 92 via thesupport plate 90 and the load sensors 94 a to 94 d. As indicated byarrows in FIG. 7, a reaction from a floor, which is generated by theinput load, is transferred from the legs 92 through the spherical parts92 a, the load sensors 94 a to 94 d and the intermediate plate 90 c tothe upper plate 90 a.

Each load sensor 94 is, for example, a strain gauge (strain sensor) loadcell, which is a load converter for converting an input load to anelectrical signal. In the load sensor 94, a strain-generating body 95 isdeformed according to an input load, resulting in a strain. The strainis converted into a change of electrical resistance and then convertedinto a change of voltage by a strain sensor 96 attached to thestrain-generating body. 95 Therefore, the load sensor 94 outputs, froman output terminal thereof, a voltage signal indicating the input load.

The load sensor 94 may be of other types, such as a tuning fork type, astring vibration type, an electrostatic capacitance type, apiezoelectric type, a magnetostrictive type, and a gyroscopic type.

Referring back to FIG. 6, the board type controller 9 further includes apower button 9 c. When the power button 9 c is operated (e.g., when thepower button 9 c is pressed) in the state where the board typecontroller 9 is not activated, power is supplied to each of circuitcomponents (see FIG. 8) of the board type controller 9. However, thereare cases in which the board type controller 9 is powered on inaccordance with an instruction from the game apparatus body 5 andthereby supply of power to the circuit components is started. The boardtype controller 9 may be automatically powered off when a state wherethe user does not stand thereon continues for a predetermined period oftime (e.g., 30 sec) or more. Further, when the power button 9 c is againoperated in the state where the board type controller 9 is in the activestate, the board type controller 9 may be powered off to stop supply ofpower to the circuit components.

FIG. 8 is a block diagram illustrating an example of an electricalconfiguration of the board type controller 9. In FIG. 8, flows ofsignals and data are indicated by solid arrows, and supply of power isindicated by dotted arrows.

As shown in FIG. 8, the board type controller 9 includes a microcomputer100 for controlling the operation thereof. The microcomputer 100includes a CPU, a ROM, a RAM, and the like, which are not shown. The CPUcontrols the operation of the board type controller 9 in accordance witha program stored in the ROM.

The power button 9 c, an AD converter 102, a DC-DC converter 104, and awireless module 106 are connected to the microcomputer 100. An antenna106 a is connected to the wireless module 106. The four load sensors 94a to 94 b are connected to the AD converter 102 via amplifiers 108,respectively.

Further, the board type controller 9 includes a battery 110 forsupplying power to the circuit components. In another embodiment, an ACadapter may be connected to the board type controller 9 instead of thebattery 110 so that commercial power is supplied to the circuitcomponents. In this case, instead of the DC-DC converter 104, a powercircuit, which converts alternating current into direct current andlowers and rectifies a direct-current voltage, may be provided in theboard type controller 9. In the exemplary embodiment, power is supplieddirectly from the battery 110 to the microcomputer 100 and the wirelessmodule 106. In other words, power is constantly supplied from thebattery 110 to the wireless module 106 and some components (such as theCPU) in the microcomputer 100 to detect whether or not the power button9 c is turned on and whether or not a command that instructs power-on istransmitted from the game apparatus body 5. On the other hand, power issupplied from the battery 110 through the DC-DC converter 104 to theload sensors 94 a to 94 d, the AD converter 102, and the amplifiers 108.The DC-DC converter 104 converts a voltage value of direct currentsupplied from the battery 110 into a different voltage value, andsupplies the resultant direct current to the load sensors 94 a to 94 d,the AD converter 102, and the amplifiers 108.

Supply of power to the load sensors 94 a to 94 d, the A/D converter 102and the amplifiers 108 may be performed according to need by themicrocomputer 100 that controls the DC-DC converter 104. Specifically,when the microcomputer 100 determines that it is necessary to operatethe load sensors 94 a to 94 d to detect a load, the microcomputer 100may control the DC-DC converter 104 to supply power to the load sensors94 a to 94 d, the A/D converter 102 and the amplifiers 108.

When power is supplied to the load sensors 94 a to 94 d, the loadsensors 94 a to 94 d each output a signal indicating a load inputtedthereto. These signals are amplified by the respective amplifiers 108,and converted from analog signals into digital data by the A/D converter102. The digital data is input to the microcomputer 100. The detectedvalues of the load sensors 94 a to 94 d are given identificationinformation of the load sensors 94 a to 94 d, so that the load sensors94 a to 94 d can be identified from the corresponding detected values.Thus, the microcomputer 100 can obtain the data representing thedetected load values of the four load sensors 94 a to 94 d at the sametime.

On the other hand, when the microcomputer 100 determines that it is notnecessary to operate the load sensors 94 a to 94 d, i.e., when it is notthe time for load detection, the microcomputer 100 controls the DC-DCconverter 104 to stop supply of power to the load sensors 94 a to 94 d,the A/D converter 102, and the amplifiers 108. Thus, the board typecontroller 9 can operate the load sensors 94 a to 94 d to detect a loador a distance only when it is required, resulting in a reduction inpower consumption for load detection.

Load detection is typically required when the game apparatus body 5(FIG. 1) needs to obtain load data. For example, when game apparatusbody 5 requires load information, the game apparatus body 5 transmits aninformation acquisition command to the board type controller 9. When themicrocomputer 100 receives the information acquisition command from thegame apparatus body 5, the microcomputer 100 controls the DC-DCconverter 104 to supply power to the load sensors 94 a to 94 d and thelike, thereby detecting a load. On the other hand, when themicrocomputer 100 does not receive a load acquisition command from thegame apparatus body 5, the microcomputer 100 controls the DC-DCconverter 104 to stop supply of power to the load sensors 94 a to 94 dand the like.

The microcomputer 100 may control the DC-DC converter 104 based on adetermination that load detection timing occurs at predeterminedintervals. When such periodic load detection is performed, informationregarding the constant time period may be supplied and stored from thegame apparatus body 5 to the microcomputer 100 of the board typecontroller 9 when the game is started, or it may be preinstalled in themicrocomputer 100.

The data representing the detected values from the load sensors 94 a to94 d are transmitted as board operation data (input data) for the boardtype controller 9 from the microcomputer 100 via the radio module 106and an antenna 106 b to the game apparatus body 5. For example, when themicrocomputer 100 has performed load detection according to a commandfrom the game apparatus body 5, the microcomputer 100 transmits thedetected value data of the load sensors 94 a to 94 d to the gameapparatus body 5 on receipt of the detected value data from the A/Dconverter 102. The microcomputer 100 may transmit the detected valuedata to the game apparatus body 5 at predetermined intervals. If theinterval of the data transmission is longer than the interval of theload detection, data containing load values which have been detected ata plurality of detection timings up to the next transmission timing maybe transmitted.

The wireless module 106 is set so as to perform communication accordingto the same wireless standard (the Bluetooth, wireless LAN, and thelike) as that for the controller communication module 19 of the gameapparatus body 5. Accordingly, the CPU 10 of the game apparatus body 5is allowed to transmit an information acquisition command to the boardtype controller 9 through the controller communication module 19 and thelike. Thus, the board type controller 9 is allowed to receive thecommand from the game apparatus body 5 through the wireless module 106and the antenna 106 a. Further, the board type controller 9 is allowedto transmit the board operation data including the load detection values(or load calculation values) of the load sensors 94 a to 94 d to thegame apparatus body 5.

For example, in a game which is executed based on a simple sum of fourload values detected by the four load sensors 94 a to 94 d, the user isallowed to stand at any position with respect to the four load sensors94 a to 94 d of the board type controller 9. That is, the user isallowed to stand on the platform 9 a at any position and in anydirection to play a game. In some kinds of games, however, the directionof a load value detected by each of the four load sensors 94 viewed fromthe user needs to be identified. That is, a positional relation betweenthe four load sensors 94 of the board type controller 9 and the userneeds to be recognized. In this case, for example, the positionalrelation between the four load sensors 94 and the user may be previouslydefined, and the user may be supposed to stand on the platform 9 a in amanner which allows the predetermined positional relation. Typically, apositional relation in which two of the load sensors 94 a to 94 d arepresent in front of, behind, to the right of, and to the left of theuser standing in the center of the platform 9 a, i.e., a positionalrelation in which the user stands in the center of the platform 9 a ofthe board type controller 9, is defined. In this case, the platform 9 aof the board type controller 9 is rectangular in shape as viewed fromthe top, and the power button 9 c is provided at one side (long side) ofthe rectangle. Therefore, it is previously ruled that the user, usingthe power button 9 c as a guide, stands on the platform 9 a such thatthe long side at which the power button 9 c is provided is located in apredetermined direction (front, rear, left or right). In this case, eachof the load values detected by the load sensors 94 a to 94 d is a loadvalue of a predetermined direction (front right, front left, rear right,or rear left) as viewed from the user. Therefore, the board typecontroller 9 and the game apparatus body 5 can find out a direction towhich each detected load value corresponds as viewed from the user,based on the identification information of the load sensors 94 containedin the detected load value data and arrangement data representingpreviously set (stored) positions or directions of the load sensors 94with respect to the user. As a result, it is possible to recognize whatis intended by a game operation performed by the user, such as anoperation direction of forward, backward, rightward or leftward, orwhich foot of the user is lifted.

Next, an outline of information processing performed on the gameapparatus body 5 will be described with reference to FIGS. 9, 10A, and10B before a specific description of processes performed by the gameapparatus body 5 is given. In the following description of the outlineof information processing, a first exemplary game is used as an exampleof information processing. FIG. 9 is a diagram illustrating an exampleof a user who plays the first exemplary game by using the terminaldevice 6 and the board type controller 9. FIG. 10A is a diagramillustrating an example of an image of the first exemplary game, whichis displayed on the LCD 61 of the terminal device 6. FIG. 10B is adiagram illustrating an example of an image of the first exemplary game,which is displayed on the monitor 2.

As shown in FIG. 9, in the first exemplary game, the user performs anoperation by using the terminal device 6 and the board type controller9. Specifically, the user stands on the board type controller 9 with theterminal device 6 in his/her hands, and performs an operation. Then, theuser performs an action (e.g., stepping or knee bending) on the boardtype controller 9 and simultaneously performs an operation to move theterminal device 6, while viewing the image displayed on the monitor 2and the image displayed on the LCD 61 of the terminal device 6, therebyplaying the game. On each of the LCD 61 of the terminal device 6 and themonitor 2, a game image is displayed, in which a player object Poperforms an action (e.g., an action to change its attitude or an actionto change its movement direction) in a virtual world in accordance withthe direction and the attitude of the terminal device 6 held by the useras well as the user's action on the board type controller 9, and theposition and the attitude of a virtual camera set in the virtual worldare changed in accordance with the position and the attitude of theplayer object Po.

As shown in FIG. 10A, on the LCD 61 of the terminal device 6, the playerobject Po moving in the virtual world is displayed. In the example shownin FIG. 10A, a virtual camera is located near the back of the playerobject Po swimming in the sea, and a state of the virtual world viewedfrom the virtual camera is displayed together with player object Po.Displaying, on the LCD 61, the virtual world viewed at close range frombehind the player object Po allows the user holding the terminal device6 to experience a sense of presence in the virtual world, and allows theuser to intuitively know the movement direction and the movement speedof the player object Po. Further, the attitude and the movementdirection of the player object Po are changed when the attitude of theterminal device 6 is changed or the terminal device 6 is rotated tochange its direction, and the position and the attitude of the virtualcamera are changed in accordance with the attitude change of theterminal device 6. For example, by associating the attitude and thedirection of the terminal device 6 with the attitude and the directionof the virtual camera, the user is allowed to experience a sense oflooking into the virtual world through the LCD 61 of the terminal device6.

Further, as shown in FIG. 10B, the same virtual world as that displayedon the LCD 61 is displayed on the monitor 2. In the example shown inFIG. 10B, a state of the virtual world in which the player object Poswimming in the sea is viewed from a distance in a manner of bird's eyeviewing is displayed together with the player object Po. Further, in theexample shown in FIG. 10B, a movement trajectory Lp of the player objectPo in the sea is displayed in the virtual world. Displaying, on themonitor 2, the virtual world in which the player object Po is viewedfrom a distance in a manner of bird's eye viewing allows the user toeasily know the surroundings of the player object Po, and allows anotherperson who watches the user's playing the first exemplary game to enjoya view in which the player object Po moves in the virtual world.

For example, when the user performs stepping on the board typecontroller 9, the player object Po swims in the sea with flutter kicksat a speed based on the user's stepping action. When the user performsknee bending on the board type controller 9, the player object Po swimsin the sea with dolphin kicks at a speed based on the user's kneebending action. In this way, the user is allowed to change the swimmingstyle or the movement speed of the player object Po in accordance withhis/her action on the board type controller.

For example, as described above, detected load values based on theuser's action on the board type controller 9 are output from the boardtype controller 9. Using the detected load values allows calculation ofa total load applied to the board type controller 9, and acenter-of-gravity position of the load applied to the board typecontroller 9. Further, using a change in the total load or a change inthe center-of-gravity position allows estimation of the action of theuser on the board type controller 9. Based on the estimated user'saction on the board type controller 9, the swimming style and themovement speed of the player object Po are set.

Further, the attitude and the movement direction of the player object Poswimming in the sea change in accordance with the direction and theattitude of the terminal device 6 held by the user. For example, whenthe user turns the terminal device 6 to the upward, downward, leftward,or rightward direction, the attitude of the player object Po swimming inthe sea (the direction to which the player object Po faces in thevirtual world) changes in accordance with the direction change of theterminal device 6, and the direction in which the player object Po swimsalso changes. Specifically, when the user changes the direction of theterminal device 6 such that the back surface of the terminal device 6faces upward, the attitude of the player object Po changes such that theplayer object Po faces the sea surface, and the swimming direction ofthe player object Po changes such that the player object Po swims upwardto the sea surface. When the user changes the direction of the terminaldevice 6 such that the back surface of the terminal device 6 facesleftward, the attitude of the player object Po changes to the left asviewed from the player object Po, and the swimming direction of theplayer object Po changes to the left. In this way, the user is allowedto change the attitude and the movement direction of the player objectPo in accordance with the direction and the attitude of the terminaldevice 6 held by the user. For example, by associating the attitude andthe direction of the terminal device 6 with the attitude and themovement direction of the player object Po, the user is allowed toperform, by using the terminal device 6, a realistic operation as ifhe/she becomes the player object Po. Further, since the virtual camerais located near the back of the player object Po swimming in the sea,when the attitude and the movement direction of the player object Pochange, the attitude and the position of the virtual camera also change.For example, by associating the attitude and the direction of theterminal device 6 with the attitude and the direction of the virtualcamera, the user is allowed to experience a sense of looking into thevirtual world through the LCD 61 of the terminal device 6.

For example, acceleration data or angular velocity data based on achange in the attitude of the terminal device 6 is output from theterminal device 6. Since the direction of the gravitational accelerationapplied to the terminal device 6 can be calculated by using theacceleration represented by the acceleration data, it is possible toestimate the attitude of the terminal device 6 with respect to thevertical direction in the real space. Further, since the angularvelocity and/or the dynamic acceleration applied to the terminal device6 are known by using the angular velocity represented by the angularvelocity data and/or the acceleration represented by the accelerationdata, it is possible to estimate a change in the attitude (a change inthe direction) of the terminal device from its initial attitude in thereal space by using the angular velocity and/or the dynamicacceleration. Based on the estimated attitude change (direction change)of the terminal device, the attitude and the movement direction of theplayer object Po as well as the attitude and the position of the virtualcamera are set.

Next, a process of the first embodiment, which is performed in the gamesystem 1, will be described in detail. First, main data used in theprocess will be described with reference to FIG. 11. FIG. 11 is adiagram illustrating an example of main data and programs stored in theexternal main memory 12 and/or the internal main memory 35 (hereinafter,these two main memories are collectively referred to as a main memory)of the game apparatus body 5 of the first embodiment.

As shown in FIG. 11, in a data storage area of the main memory, boardoperation data Daa, terminal operation data Dab, load value data Dac,center-of-gravity position data Dad, terminal device direction/attitudedata Dae, operation direction data Daf, action/attitude data Dag,movement vector data Dah, position data Dai, virtual camera data Daj,movement trajectory data Dak, action mode flag data Dam, image data Dan,and the like are stored. In addition to the data shown in FIG. 11, datarequired for a game process, such as image data of various objectsdisplayed on the monitor 2 and the LCD 61, and sound data used for agame, are optionally stored in the main memory. In a program storagearea of the main memory, various programs Pa constituting an informationprocessing program of the first embodiment are stored.

As the board operation data Daa, a series of operation information(board operation data) transmitted as transmission data from the boardtype controller 9 is stored, and updated to the latest board operationdata. For example, the board operation data Daa includes load data Daa1and the like. The load data Daa1 is data representing load valuesdetected by the load sensors 94 a to 94 d of the board type controller9.

As the terminal operation data Dab, a series of operation information(terminal operation data) transmitted as transmission data from theterminal device 6 is stored, and updated to the latest terminaloperation data. For example, the terminal operation data Dab includesacceleration data Dab1, angular velocity data Dab2, and the like. Theacceleration data Dab1 is data representing an acceleration (anacceleration vector) detected by the acceleration sensor 603. Forexample, the acceleration data Dab1 represents a three-dimensionalacceleration vector whose components are accelerations in the three-axis(x-axis, y-axis, and z-axis) directions shown in FIG. 3. In anotherembodiment, the acceleration data Dab1 may represent an acceleration inany one or more directions. The angular velocity data Dab2 is datarepresenting an angular velocity detected by the gyro sensor 604. Forexample, the angular velocity data Dab2 represents angular velocitiesaround the three axes (x-axis, y-axis, and z-axis) shown in FIG. 3. Inanother example, the angular velocity data Dab2 may represent an angularvelocity around any one or more axes.

The game apparatus body 5 sequentially receives the data (e.g., the datarepresenting the detected load value, the acceleration, and the angularvelocity) included in the operation information transmitted from thecontroller 7, the board type controller 9, and the terminal device 6 atpredetermined intervals (e.g., at intervals of 1/200 sec.). For example,the received data is successively stored in the main memory by the I/Oprocessor 31. In a process flow described later, the CPU 10 reads, everyframe period (e.g., 1/60 sec.), the latest board operation data and thelatest terminal operation data from the main memory, to update the boardoperation data Daa and the terminal operation data Dab, respectively.

The operation information transmitted at predetermined intervals fromthe controller 7, the board type controller 9, and the terminal device 6may be temporarily stored in the buffer (not shown) included in thecontroller communication module 19 or the terminal communication module28. In this case, the data stored in the buffer is read every frameperiod, and thus the board operation data Daa (e.g., the load data Daa1)or the terminal operation data Dab (e.g., the acceleration data Dab1 andthe angular velocity data Dab2) in the main memory is updated to beused. At this time, since the cycle of receiving the operationinformation is different from the processing cycle, a plurality ofpieces of information received at a plurality of timings are stored inthe buffer. The process is executed by using only the latest operationinformation among the plurality of pieces of operation informationreceived at the plurality of timings.

The load value data Dac is an aggregate of data each representing a loadvalue detected by the board type controller 9. For example, the loadvalue data Dac is an aggregate of data each representing a sum of loadvalues (total load value) detected by the load sensors 94 a to 94 d.Specifically, the load value data Dac is an array of data eachrepresenting the above-mentioned total load value within a predeterminedperiod, which is calculated time-sequentially, and the data representingthe total load value is time-sequentially stored in each element of thearray.

The center-of-gravity position data Dad is an aggregate of data eachrepresenting a center-of-gravity position of a load applied to the boardtype controller 9. For example, the center-of-gravity position data Dadis an aggregate of data each representing a position of center ofgravity which is calculated based on the load values detected by therespective load sensors 94 a to 94 d by using a predetermined numericalformula. Specifically, the center-of-gravity position data Dad is anarray of data each representing the above-mentioned center-of-gravityposition within a predetermined period, which is calculatedtime-sequentially, and the data representing the center-of-gravityposition is time-sequentially stored in each element of the array.

The terminal device direction/attitude data Dae includes real spacereference direction data Dae1, current direction data Dae2, and thelike. The real space reference direction data Dae1 is data representinga reference direction (attitude; real space reference direction) of theterminal device 6 in the real space. The current direction data Dae2 isdata representing a direction and an attitude (current direction) of theterminal device 6 at the current time in the real space. For example, areal space reference direction data Dae1 and a current direction dataDae2 are calculated based on the acceleration data Dab1 and the angularvelocity data Dab2 included in the terminal operation data Dab. Methodsfor calculating a real space reference direction and a current directionwill be described later.

The operation direction data Daf includes virtual world referencedirection data Daf1, operation instruction direction data Daf2, and thelike. The virtual world reference direction data Daf1 is datarepresenting a reference direction set in the virtual world. Theoperation instruction direction data Daf2 is data representing anoperation instruction direction which is currently instructed in thevirtual world by the user. Methods for calculating a virtual worldreference direction and an operation instruction direction will bedescribed later.

The action/attitude data Dag is data representing an action and anattitude of the player object Po in the virtual world. The movementvector data Dah is data representing a movement speed and a movementdirection of the player object Po in the virtual world. For example, themovement vector data Dah represents a movement vector in the virtualworld. The position data Dai is data representing a position of theplayer object Po in the virtual world. The virtual camera data Daj isdata relating to a virtual camera set in the virtual world. For example,the virtual camera data Daj includes data relating to a first virtualcamera for generating a game image to be displayed on the LCD 61 of theterminal device 6, and data relating to a second virtual camera forgenerating a game image to be displayed on the monitor 2. The movementtrajectory data Dak is data representing a trajectory along which theplayer object Po has moved in the virtual world.

The action mode flag data Dam is data representing an action mode flagwhich is set in accordance with an action (e.g., a swimming style) seton the player object Po. For example, when the user performs kneebending on the board type controller 9, the action of the player objectPo is set to an action of swimming with dolphin kicks in the sea, andthe action mode flag is turned on.

The image data Dan includes player object data Dan1, background imagedata Dan2, and the like. The player object data Dan1 is data forarranging the player object Po of the first exemplary game in thevirtual world to generate a game image. The background image data Dan2is data for arranging a background of the first exemplary game in thevirtual world to generate a game image.

Next, the process of the first embodiment performed by the gameapparatus body 5 will be described in detail with reference to FIGS. 12to 17. FIG. 12 is a flowchart illustrating an example of a process ofthe first embodiment executed by the game apparatus body 5. FIG. 13 is asub-routine illustrating an example of a game control process in step 44in FIG. 12. FIG. 14 is a sub-routine illustrating an example of a playerobject setting process in step 83 in FIG. 13. FIG. 15 is a sub-routineillustrating an example of a player object action setting process instep 126 in FIG. 14. FIG. 16 is a diagram illustrating an example of arelationship between a real space reference direction and a currentdirection which are projected on a horizontal plane in a real space, anda virtual world reference direction and an operation instructiondirection which are projected on a horizontal plane in a virtual world.FIG. 17 is a diagram illustrating an example of a relationship between acurrent direction projected on a vertical plane in the real space, andan operation instruction direction projected on a vertical plane in thevirtual world. In the flowcharts shown in FIGS. 12 to 15, process stepsfor causing the player object Po to move and displaying the same, inaccordance with an operation performed by a user using the terminaldevice 6 and the board type controller 9 will be mainly described amongthe whole process, while detailed description of other process steps isnot given. Further, in FIGS. 12 to 15, each of the steps executed by theCPU 10 is abbreviated as “S”.

When the game apparatus body 5 is powered on, the CPU 10 of the gameapparatus body 5 executes a boot program stored in the ROM/RTC 13 toinitialize each unit such as the main memory. Then, the informationprocessing program stored in the optical disc 4 is loaded to the mainmemory, and the CPU 10 starts to execute the program. The flowchartsshown in FIGS. 12 to 15 show process steps to be performed after theabove-mentioned process steps are completed.

As shown in FIG. 12, the CPU 10 performs an initialization process (step40), and proceeds to the next step. For example, in the initializationprocess at step 40, the CPU 10 constructs a virtual world, arranges aplayer object Po and other objects, which appear in a game space, intheir initial positions, and sets the initial values of variousparameters used for the game process.

Next, the CPU 10 sets a reference direction based on data transmittedfrom the terminal device 6 (step 41), and proceeds to the next step.Hereinafter, an example of setting of a reference direction by the CPU10 will be described.

The terminal device 6 repeatedly transmits the above-mentioned data tothe game apparatus body 5. In the game apparatus body 5, the terminalcommunication module 28 sequentially receives the above data, and theI/O processor 31 sequentially stores the terminal operation data, thecamera image data, and the microphone sound data in the main memory. Instep 41, the CPU 10 reads the latest terminal operation data from themain memory, and updates the acceleration data Dab1 and the angularvelocity data Dab2.

Next, the CPU 10 calculates a direction and an attitude of the terminaldevice 6 in the real space. For example, the CPU 10 calculates, as areference direction (initial attitude) in the real space, a directionand an attitude of the terminal device 6 at the current time, based onthe acceleration represented by the acceleration data Dab1 and theangular velocity represented by the angular velocity data Dab2, andupdates the real space reference direction data Dae1 by using datarepresenting the calculated reference direction of the terminal device6. For example, the CPU 10 is capable of calculating an amount ofrotation (an amount of direction change) of the terminal device 6 in thereal space, per unit time, based on the angular velocity represented bythe angular velocity data Dab2. Further, in a state where the terminaldevice 6 is substantially stationary (in a static state) in the realspace, the acceleration applied to the terminal device 6 is thegravitational acceleration. Therefore, the CPU 10 is capable ofcalculating a direction of gravity applied to the terminal device 6(i.e., an attitude of the terminal device 6 with respect to the verticaldirection in the real space), based on the acceleration represented bythe acceleration data Dab1. Accordingly, the CPU 10 is capable ofcalculating an initial attitude of the terminal device 6, based on theacceleration represented by the acceleration data Dab1 and the angularvelocity represented by the angular velocity data Dab2. In the followingdescription, when step 41 is executed, a direction in which the backsurface of the terminal device 6 faces in the real space (the z-axispositive direction shown in FIG. 3) is the real space referencedirection.

The initial attitude of the terminal device 6 may be calculated based onthe acceleration represented by the acceleration data Dab1, or on thedirection of a magnetism detected by the magnetic sensor 602.Alternatively, the user is caused to perform a predetermined operationwith the terminal device 6 being set in a specific attitude, and thespecific attitude at the time when the predetermined operation isperformed may be used as an initial attitude. As for the timing at whichthe initial attitude is set, i.e., the timing at which step 41 isexecuted, step 41 may be automatically executed when the game isstarted, or may be executed in accordance with that the user performs apredetermined operation (e.g., an operation to press a predeterminedoperation button 64) by using the terminal device 6.

In step 41, the CPU 10 calculates a current direction in which theplayer object Po faces in the virtual world (e.g., a direction in whichthe player object Po moves if there is no direction change) as areference direction (initial attitude) in the virtual world, and updatesthe virtual world reference direction data Daf1 by using datarepresenting the calculated reference direction in the virtual world.

The above-mentioned step 41 is followed by a process at step 42.Thereafter, a process loop of a series of process steps 42 to 51 isrepeatedly executed every predetermined period (one frame period).

In step 42, the CPU 10 obtains board operation data transmitted from theboard type controller 9, and proceeds to the next step. The board typecontroller 9 repeatedly transmits the board operation data to the gameapparatus body 5. Accordingly, in the game apparatus body 5, thecontroller communication module 19 sequentially receives the boardoperation data, and the I/O processor 31 sequentially stores thereceived board operation data in the main memory. The interval oftransmission of the board operation data from the board type controller9 may be shorter than the game processing period (one frame period), andfor example, it is 1/200 sec. In step 42, the CPU 10 reads the latestboard operation data from the main memory to update the board operationdata Daa1. The board operation data includes data representingidentification information of the load sensors 94 a to 94 d, and datarepresenting the load values detected by the load sensors 94 a to 94 d.The load data Daa1 is updated by using each of the data identified bythe identification information.

Next, the CPU 10 obtains various data transmitted from the terminaldevice 6 (step 43), and proceeds to the next step. The terminal device 6repeatedly transmits the data to the game apparatus body 5. Accordingly,in the game apparatus body 5, the terminal communication module 28sequentially receives the data, and the codec LSI 27 sequentiallyperforms a decompression process on the camera image data and themicrophone sound data. Then, the I/O processor 31 sequentially storesthe terminal operation data, the camera image data, and the microphonesound data in the main memory. In step 43, the CPU 10 reads the latestterminal operation data from the main memory to update the accelerationdata Dab1 and the angular velocity data Dab2.

Next, the CPU 10 performs a game control process (step 44), and proceedsto the next step. The game control process is a process in which, forexample, the player object Po in the virtual world is caused to move inaccordance with a game operation performed by the user, therebyprogressing the game. In the first exemplary game, the user is allowedto play various games by using the terminal device 6 and the board typecontroller 9. Hereinafter, the game control process at step 44 will bedescribed with reference to FIG. 13.

In FIG. 13, the CPU 10 calculates a load value and a center-of-gravityposition (step 81), and proceeds to the next step. For example, the CPU10 calculates a total load value by summing up the detected load valuesrepresented by the load data Daa1, and updates the latest data in thetime-sequence data array of the load value data Dac by using the datarepresenting the total load value. Specifically, since the load dataDaa1 represents the latest load values detected by the load sensors 94 ato 94 d, the total load value is obtained by summing up the detectedload values. Further, the CPU 10 calculates a center-of-gravity positionby using the detected load values represented by the load data Daa1, andupdates the latest data in the time-sequence data array of thecenter-of-gravity position data Dad by using the data representing thecenter-of-gravity position. Hereinafter, an example of acenter-of-gravity position calculation method will be described.

The center-of-gravity position is a position of the center of gravity ofa load applied to the platform 9 a of the board type controller 9, andis determined based on the load values detected by the load sensors 94 ato 94 d (see FIG. 6). For example, the center-of-gravity position isrepresented by coordinate values based on a predetermined coordinatesystem corresponding to a position on the platform 9 a of the board typecontroller 9 (e.g., a X1Y1 coordinate system in which the center of theplatform 9 a corresponds to an origin thereof, the long side directionof the platform 9 a corresponds to the X1 axis direction, and the shortside direction corresponds to the Y1 axis direction). Assuming that aload value detected by the load sensor 94 a is defined as “a”, a loadvalue detected by the load sensor 94 b is defined as “b”, a load valuedetected by the load sensor 94 c is defined as “c”, and a load valuedetected by the load sensor 94 d is defined as “d”, an X1-axiscoordinate value (X1) and a Y-axis coordinate value (Y1) of the centerof gravity are calculated in accordance with the following equations:

X1=((a+c)−(b+d))×m

Y1=((c+d)−(a+b))×n

where m and n are predetermined constants.

The total load value and the center-of-gravity position, which are thuscalculated, vary in accordance with an action or a weight shift(posture) of the user on the board type controller 9. For example, whenthe user repeats stepping on the board type controller 9, the total loadvalue varies in accordance with the stepping cycle, and thecenter-of-gravity position shifts as if reciprocating, according to thestepping cycle, between the position on which the user puts his/her leftfoot and the position on which the user puts his/her right foot. Inanother example, when the user repeats knee bending on the board typecontroller 9, the total load value varies in accordance with the cycleof the knee bending, but the variation of the center-of-gravity positionis relatively reduced.

Next, the CPU 10 calculates a direction change and an attitude of theterminal device 6 (step 82), and proceeds to the next step. For example,the CPU 10 calculates a direction change and an attitude of the terminaldevice 6, based on the acceleration represented by the acceleration dataDab1 and the angular velocity represented by the angular velocity dataDab2, and updates the current direction data Dae2 by using datarepresenting the current direction which corresponds to the calculateddirection change and attitude of the terminal device 6. The CPU 10 iscapable of calculating an amount of rotation (an amount of directionchange) of the terminal device 6 in the real space, per unit time, basedon the angular velocity represented by the angular velocity data Dab2.Further, in a state where the terminal device 6 is substantiallystationary (in a static state) in the real space, the accelerationapplied to the terminal device 6 is the gravitational acceleration.Therefore, the CPU 10 is capable of calculating a direction of gravityapplied to the terminal device 6 (i.e., the attitude of the terminaldevice 6 with respect to the vertical direction in the real space),based on the acceleration represented by the acceleration data Dab1.Accordingly, the CPU 10 is capable of calculating a direction change andan attitude of the terminal device 6, based on the accelerationrepresented by the acceleration data Dab1 and the angular velocityrepresented by the angular velocity data Dab2. In the followingdescription, a direction (the z-axis positive direction shown in FIG. 3)at the current time, in which the back surface of the terminal device 6faces in the real space, is regarded as the current direction of theterminal device 6.

In the exemplary embodiment, a direction change and an attitude of theterminal device 6 are calculated based on the data representing theacceleration and the angular velocity detected by the terminal device 6.In another embodiment, however, a direction change and an attitude ofthe terminal device 6 may be calculated by using any one piece data orthree or more pieces of data. For example, the magnetic sensor 602included in the terminal device 6 detects a geomagnetism applied to theterminal device 6. Therefore, based on the direction of the geomagnetismapplied to the terminal device 6, a predetermined orientation withrespect to the terminal device 6 (i.e., the attitude of the terminaldevice 6 with respect to the predetermined orientation) can becalculated. Even when a magnetic field is generated in addition to thegeomagnetism in the real space where the terminal device 6 is located,an amount of rotation of the terminal device 6 can be calculated.Accordingly, the CPU 10 is capable of calculating a direction change andan attitude of the terminal device 6 by using at least one of the datarepresenting the acceleration, the data representing the angularvelocity, and the data representing the magnetism, which are detected bythe terminal device 6.

Any calculation method may be used for calculating an attitude of theterminal device 6. For example, a calculation method is considered, inwhich the attitude of the terminal device 6, which is calculated basedon the angular velocity represented by the angular velocity data Dab2,is corrected by using the acceleration represented by the accelerationdata Dab1 and the direction of the magnetism detected by the magneticsensor 602.

Specifically, the CPU 10 firstly calculates an attitude of the terminaldevice 6 based on the angular velocity represented by the angularvelocity data Dab2. Any method may be used for calculating an attitudeof the terminal device 6 based on the angular velocity. For example, anattitude of the terminal device 6 may be calculated by using theimmediately preceding attitude (most-recently calculated attitude) andthe current angular velocity (the angular velocity currently obtained atstep 42 in the process loop). The CPU 10 causes the immediatelypreceding attitude to rotate at the current angular velocity for a unittime to obtain a new attitude. The immediately preceding attitude isrepresented by the current direction data Dae2, and the current angularvelocity is represented by the angular velocity data Dab2. Accordingly,the CPU 10 reads the current direction data Dae2 and the angularvelocity data Dab2, and calculates an attitude of the terminal device 6.As described above, the initial attitude of the terminal device 6 isdetermined in step 41. Therefore, when calculating an attitude of theterminal device 6 based on the angular velocity, the CPU 10 cancalculate an attitude of the terminal device 6 at the current time,based on the initial attitude of the terminal device 6.

Next, the CPU 10 corrects the attitude of the terminal device 6calculated based on the angular velocity, by using the accelerationrepresented by the acceleration data Dab1. Specifically, the CPU 10calculates an attitude of the terminal device 6 based on theacceleration represented by the acceleration data Dab1. In a state wherethe terminal device 6 is substantially stationary, the accelerationapplied to the terminal device 6 is the gravitational acceleration.Accordingly, in this state, the CPU 10 is capable of calculating adirection of the gravitational acceleration (direction of gravity) byusing the direction of the acceleration represented by the accelerationdata Dab1, and therefore, is capable of calculating a direction(attitude) of the terminal device 6 with respect to the direction ofgravity.

After the calculation of the attitude of the terminal device 6 based onthe acceleration, the CPU 10 corrects the attitude based on the angularvelocity, by using the attitude based on the acceleration. Specifically,the CPU 10 performs a correction to approximate, at a predeterminedrate, the attitude of the terminal device 6 calculated based on theangular velocity to the attitude of the terminal device 6 calculatedbased on the acceleration. The predetermined rate may be a preset fixedvalue, or may be set according to, for example, the accelerationrepresented by the acceleration data Dab1. Regarding the attitude of theterminal device 6 calculated based on the acceleration, since anattitude cannot be calculated with respect to the direction of rotationaround the direction of gravity, the CPU 10 may not perform correctionof the attitude with respect to this rotation direction. When correctingthe attitude of the terminal device 6 calculated based on the angularvelocity, based on the direction of magnetism detected by the magneticsensor 602, the CPU 10 approximates, at a predetermined rate, theattitude of the terminal device 6 calculated based on the angularvelocity to the attitude of the terminal device 6 calculated based onthe direction of magnetism. Thus, the CPU 10 is capable of accuratelycalculating an attitude of the terminal device 6.

Next, the CPU 10 sets a player object Po (step 83), and proceeds to thenext step. Hereinafter, a player object setting process in step 83 willbe described with reference to FIG. 14.

In FIG. 14, the CPU 10 calculates a horizontal angular differencebetween the real space reference direction and the current direction(step 120), and proceeds to the next step. The horizontal angulardifference is an angular difference (e.g., an angle A shown in FIG. 16)which is obtained by projecting an angular difference between the realspace reference direction and the current direction in the real space ona horizontal plane. The horizontal angular difference indicates an angleat which the direction of the terminal device 6 (the direction in whichthe back surface of the terminal device 6 faces (the z-axis positivedirection shown in FIG. 3)) is changed from the initial attitude of theterminal device 6 around the vertical direction in the real space. Forexample, the CPU 10 calculates the horizontal angular difference byusing the real space reference direction represented by the real spacereference direction data Dae1, and the current direction represented bythe current direction data Dae2.

Next, the CPU 10 calculates an operation instruction direction withrespect to the virtual world reference direction in accordance with thehorizontal angular difference and the attitude (up-down angle) of theterminal device 6 in the up-down direction (step 121), and proceeds tothe next step. For example, as shown in FIG. 16, the CPU 10 calculates,based on the virtual world reference direction represented by thevirtual world reference direction data Daf1, an operation instructiondirection in the virtual world such that an angular difference whichoccurs when the virtual world reference direction and the operationinstruction direction are projected on the horizontal plane in thevirtual world is equal to the horizontal angular difference, and thesame positional relation is achieved between the real space and thevirtual world (that is, if the current direction rotates to the leftwith respect to the real space reference direction, the operationinstruction direction also rotates to the left with respect to thevirtual world reference direction). Please refer to angle A shown inFIG. 16. Further, as shown in FIG. 17, the CPU 10 calculates, based onthe current direction represented by the current direction data Dae2, anoperation instruction direction in the virtual world such that anangular difference between the horizontal direction in the virtual worldand the operation instruction direction is equal to an angulardifference between the horizontal direction in the real space and thecurrent direction, and the same positional relation is achieved betweenthe real space and the virtual world (that is, if the current directionis downward with respect to the horizontal direction in the real space,the operation instruction direction is also downward with respect to thehorizontal direction in virtual world). Please refer to angle C in FIG.17. Then, the CPU 10 updates the operation instruction direction dataDaft by using the calculated operation instruction directions.

Next, the CPU 10 determines whether or not the player object Po moves(step 122). For example, when a movement vector is set in the movementvector data Dah, the CPU 10 determines that the player object Po moves.When the player object Po moves, the CPU 10 proceeds to step 123. On theother hand, when the player object Po does not move, the CPU 10 proceedsto step 129.

In step 123, the CPU 10 calculates a rotation movement angle inaccordance with the horizontal angular difference calculated at step120, and proceeds to the next step. For example, the CPU 10 calculates arotation movement angle B (angle B shown in FIG. 16) in accordance withthe following equation:

B=A×a1

where A is the horizontal angular difference, and a1 is a positiveconstant (e.g., 0.001) smaller than 1.

Next, the CPU 10 changes the virtual world reference direction inaccordance with the rotation movement angle calculated at step 123 (step124), and proceeds to the next step. For example, when the virtual worldreference direction represented by the virtual world reference directiondata Daf1 is projected on the horizontal plane in the virtual world, theCPU 10 changes the virtual world reference direction such that thevirtual world reference direction approaches the operation instructiondirection by the rotation movement angle, and updates the virtual worldreference direction data Daf1 by using the changed virtual worldreference direction.

Next, the CPU 10 sets a direction of the player object Po in the virtualworld, based on the operation instruction direction represented by theoperation instruction direction data Daf2 (step 125), and proceeds tothe next step. For example, the CPU 10 updates the action/attitude dataDag by using the operation instruction direction represented by theoperation instruction direction data Daft as the direction of the playerobject Po. Accordingly, the direction of the player object Po in thevirtual world is set based on the attitude of the terminal device 6 inthe real space. For example, when the user turns the back surface of theterminal device 6 upward, the direction of the player object Po is setat an elevation angle according to an angle at which the back surfacefaces upward with respect to the vertical direction. When the user turnsthe back surface of the terminal device 6 downward, the direction of theplayer object Po is set at a depression angle according to an angle atwhich the back surface faces downward with respect to the verticaldirection. Further, a left-right direction of the player object Po isset based on a left-right direction change of the terminal device 6 withrespect to the real space reference direction. For example, when theterminal device 6 rotates in the yaw direction (see FIG. 3) which is arotation direction around the y axis, the direction of the player objectPo also changes in accordance with the amount of the rotation of theterminal device 6. Specifically, when the terminal device 6 rotatesaround the y axis to the left as viewed from the y-axis negativedirection, the direction of the player object Po also changes to theleft as viewed from the player object Po in accordance with the amountof this rotation. When the user maintains the state where the terminaldevice 6 is rotated to the left or to the right with respective to thereal space reference direction, the virtual world reference directioncontinuously changes to the left or to the right, and therefore, theoperation instruction direction continuously changes to the left or tothe right in accordance with the change in the virtual world referencedirection when step 121 is executed. Accordingly, when the usermaintains the state where the terminal device 6 is rotated to the leftor to the right with respect to the real space reference direction, thedirection of the player object Po continuously changes to the left or tothe right. That is, even when the user desires to greatly change thedirection of the player object Po (e.g., to change the direction of theplayer object Po to the opposite direction), the user need not greatlychange the direction of the terminal device 6 to the opposite directionor the like, and thus the operability of the user is enhanced.

Next, the CPU 10 performs a player object action setting process (step126), and proceeds to the next step. Hereinafter, the player objectaction setting process at step 126 will be described with reference toFIG. 15.

In FIG. 15, the CPU 10 determines whether or not the action mode flagrepresented by the action mode flag data Dam is set to OFF (step 140).When the action mode flag is set to OFF, the CPU 10 proceeds to step141. When the action mode flag is set to ON, the CPU 10 proceeds to step148.

In step 141, the CPU 10 determines whether or not the total load valueis equal to or greater than a predetermined value. For example, the CPU10 determines whether or not the latest total load value represented bythe load value data Dac is equal to or greater than the predeterminedvalue. The predetermined value is a threshold value for determining thatthe user performs as action such as knee banding on the board typecontroller 9. For example, the threshold value is set to a valueobtained by adding a predetermined rate (e.g., 20%) to the weight of theuser on the board type controller 9. The weight of the user on the boardtype controller 9 may be calculated based on the total load value (e.g.,the weight may be an average of total load values obtained within apredetermined period), or may be previously inputted by the user. Whenthe total load value is equal to or greater than the predeterminedvalue, the CPU 10 proceeds to step 142. When the total load value issmaller than the predetermined value, the CPU 10 proceeds to step 144.

In step 142, the CPU 10 sets the object action to an action of doing adolphin kick, and proceeds to the next step. For example, when the totalload value is equal to or greater than the predetermined value, the CPU10 determines that the user performs knee bending on the board typecontroller 9. In this case, the CPU 10 sets the action of the playerobject Po to an action of swimming in the dolphin kick style, and to anaction of doing a dolphin kick, and then updates the action/attitudedata Dag by using the set action.

Next, the CPU 10 sets the action mode flag to ON (step 143), and endsthe process of this sub-routine. For example, the CPU 10 sets the actionmode flag represented by the action mode flag data Dam to ON.

On the other hand, when it is determined at step 141 that the total loadvalue is smaller than the predetermined value, the CPU 10 determineswhether or not the center-of-gravity position represented by thecenter-of-gravity position data Dad has shifted from outside theright-foot region to inside the right-foot region (step 144). When theresult of the determination is that the center-of-gravity position hasshifted from outside the right-foot region to inside the right-footregion, the CPU 10 proceeds to step 145. On the other hand, when theresult of the determination is that the center-of-gravity position hasnot shifted from outside the right-foot region to inside the right-footregion, the CPU 10 proceeds to step 146.

In the predetermined coordinate system (e.g., an X1Y1 coordinate systemin which the center of the platform 9 a corresponds to an originthereof, the long side direction of the platform 9 a corresponds to theX1 axis direction, and the short side direction corresponds to the Y1axis direction) corresponding to a position on the platform 9 a of theboard type controller 9, a region (right-foot region) to which thecenter-of-gravity position shifts when the user lifts the left foot andputs down the right foot in the stepping action, and a region (left-footregion) to which the center-of-gravity position shifts when the userlifts the right foot and puts down the left foot in the stepping action,are set. For example, the left-foot region and the right-foot region areprovided on the left-half part and the right-half part of the platform 9a (with the power button 9 c facing downward, for example),respectively, and a neutral region having a predetermined width andextending in the front-rear direction is set between the left-footregion and the right-foot region. In the process at step 144, thedetermination result is positive when the latest center-of-gravityposition represented by the center-of-gravity position data Dad shiftsfrom outside the right-foot region to inside the right-foot region, suchas when the latest center-of-gravity position shifts from inside theneutral region to inside the right-foot region.

It is considered that the user may perform stepping while turning on theboard type controller 9. In this case, if the positions of the left-footregion, the right-foot region, and the neutral region are fixed,accurate determination of the stepping action cannot be performed. Insuch a case, the left-foot region, the right-foot region, and theneutral region in the coordinate system may be shifted based on theshift of the center-of-gravity position in the coordinate system. Forexample, if the distance of shift of the center-of-gravity positionwithin a predetermined period is equal to or longer than a predeterminedlength, the neutral region is shifted such that, in the center positionof the movement trajectory of the center-of-gravity position within thepredetermined period, the neutral region extends orthogonally to themovement trajectory, and the left-foot region and the right-foot regionare shifted in accordance with the shift of the neutral region. Shiftingthe left-foot region, the right-foot region, and the neutral region inthis way enables accurate determination of the stepping action even whenthe user performs stepping while turning on the board type controller 9.

In the above-described determination of the stepping action, the CPU 10determines whether the foot with which the user takes a step on theboard type controller 9 is the left foot or the right foot. However,such distinction between the left foot and the right foot may not beperformed. For example, in the case where the process is performed suchthat the above-mentioned “left foot” is regarded as “one foot” and the“right foot” as “the other foot”, it is possible to make an instructionof an action in which the user's stepping action is synchronized withthe flutter kick action of the player object Po although there is apossibility that the foot with which the user takes a step and the footwith which the player object Po does a flutter kick may be reversed tothose described above.

In step 145, the CPU 10 sets the object action to an action of doing aflutter kick with its right foot, and ends the process of thissub-routine. For example, when the center-of-gravity position shiftsfrom outside the right-foot region to inside the right-foot region, theCPU 10 determines that the user performs stepping such that he/she liftsthe left foot and puts down the right foot on the board type controller9. In this case, the CPU 10 sets the action of the player object Po toan action of swimming in the flutter kick style, and to an action ofdoing a flutter kick with its right foot, and then updates theaction/attitude data Dag by using the set action.

In step 146, the CPU 10 determines whether or not the center-of-gravityposition represented by the center-of-gravity position data Dad shiftsfrom outside the left-foot region to inside the left-foot region. Whenthe result of the determination is that the center-of-gravity positionshifts from outside the left-foot region to inside the left-foot region,the CPU 10 proceeds to step 147. When the result of the determination isthat the center-of-gravity position does not shift from outside theleft-foot region to inside the left-foot region, the CPU 10 proceeds tostep 151

In step 147, the CPU 10 sets the object action to an action of doing aflutter kick with its left foot, and ends the process of thissub-routine. For example, when the center-of-gravity position shiftsfrom outside the left-foot region to inside the left-foot region, theCPU 10 determines that the user performs stepping such that he/she liftsthe right foot and puts down the left foot on the board type controller9. In this case, the CPU 10 sets the action of the player object Po toan action of swimming in the flutter kick style, and to an action ofdoing a flutter kick with its left foot, and then updates theaction/attitude data Dag by using the set action.

When it is determined at step 140 that the action mode flag is ON, theCPU 10 determines whether or not a predetermined time has passed fromwhen the action mode flag was set to ON (step 148). When the result ofthe determination is that the predetermined time has passed, the CPU 10proceeds to step 149. When the predetermined time has not passed, theCPU 10 proceeds to step 150. The predetermined time is a time from whena dolphin kick is done to when a next dolphin kick operation input orflutter kick operation input is enabled. Until the predetermined timehas passed, determination of the user's action by using the board typecontroller 9 is not performed. Thereby, it is possible to prevent anerroneous determination of the user's action, which might occurimmediately after a dolphin kick.

In step 149, the CPU 10 sets the action mode flag to OFF, and ends theprocess of this sub-routine. For example, the CPU 10 sets the actionmode flag represented by the action mode flag data Dam to OFF.

On the other hand, in step 150, the CPU 10 sets the object action to“during dolphin kick”, and ends the process of this sub-routine. Forexample, the CPU 10 continuously sets the action of the player object Poto an action of swimming in the dolphin kick style, and to “duringdolphin kick”, and then updates the action/attitude data Dag by usingthe set action.

In the case where the action mode flag is OFF (negative determination atstep 140), the total load value is smaller than the predetermined value(negative determination at step 141), the center-of-gravity positiondoes not shift from outside the right-foot region to inside theright-foot region (negative determination at step 144), and thecenter-of-gravity position does not shift from outside the left-footregion to inside the left-foot region (negative determination at step146), the CPU 10 sets the object action to an action of continuing thecurrently set action (step 151), and ends the process of thissub-routine. For example, when an action of doing a kick (e.g., aflutter kick) is set, the CPU 10 cancels the action, and maintains theswimming style of the player object Po, which is set as the objectaction, and then updates the action/attitude data Dag by using the setaction.

Referring back to FIG. 14, after the player object action settingprocess at step 126, the CPU 10 sets a movement vector of the playerobject Po (step 127), and proceeds to the next step. For example, theCPU 10 calculates a movement speed of the player object Po, based on thetime-sequential array of the total load values (the history of variationof the total load value) represented by the load value data Dac, and theaction (swimming style) of the player object Po which is set at step126. For example, the shorter the cycle of variation of the total loadvalue is, the higher the movement speed of the player object Po the CPU10 calculates. The greater the maximum value or the variation width ofthe total load value is, the higher the movement speed of the playerobject Po the CPU 10 calculates. That is, the shorter the pitch of theuser's stepping or knee bending on the board type controller 9 is, thehigher the movement speed of the player object Po is. The greater theforce of the user's stepping or knee bending on the board typecontroller 9 is, the higher the movement speed of the player object Pois. Then, the CPU 10 corrects the calculated movement speed by using acoefficient according to the set swimming style, thereby determining amovement speed of the player object Po. The movement speed of the playerobject Po is set so as to be relatively high at the timing of doing akick and be gradually lowered until a next kick.

Further, the CPU 10 sets, as a movement direction of the player objectPo, the operation instruction direction in the virtual world, which isrepresented by the operation instruction direction data Daf2. Then, theCPU 10 calculates a movement vector in the virtual world by using thecalculated movement speed and movement direction of the player objectPo, and updates the movement vector data Dah by using the calculatedmovement vector. For example, the CPU 10 sets a length of the movementvector by using the calculated movement speed of the player object Po,and sets a direction of the movement vector by using the operationinstruction direction represented by the operation instruction directiondata Daf2.

Next, the CPU 10 calculates a position of the player object Po (step128), and ends the process of this sub-routine. For example, the CPU 10causes the position of the player object Po represented by the positiondata Dai to shift in the virtual world, based on the movement vectorrepresented by the movement vector data Dah, and updates the positiondata Dai by using the position after the shifting.

On the other hand, when it is determined at step 122 that the playerobject Po does not move, the CPU 10 sets a direction of the playerobject Po in the virtual world, based on the operation instructiondirection represented by the operation instruction direction data Daf2(step 129), and proceeds to the next step. Since the process at step 129is identical to the process at step 125, repeated description is notnecessary.

Next, the CPU 10 sets the object action in a static state (step 130),and ends the process of this sub-routine. For example, the CPU 10 setsthe action of the player object Po in a static state (e.g., a state inwhich the player object Po is not swimming but drifting in the sea), andupdates the action/attitude data Dag by using the set action.

Referring back to FIG. 13, after the player object setting process atstep 83, the CPU 10 sets parameters relating to virtual cameras (step84), and proceeds to the next step. For example, a terminal game imageand a monitor game image are each generated as, for example, athree-dimensional CG image which is obtained by calculating a scene of agame space viewed from a virtual camera arranged in a virtual world(virtual space). Specifically, a first virtual camera for generating aterminal game image is set such that a state of the virtual world viewedat close range from behind the player object Po arranged in the virtualworld is included in the terminal game image. A second virtual camerafor generating a monitor game image is set in the same virtual world inwhich the first virtual camera is set, such that a state of the virtualworld in which the player object Po arranged in the virtual world isviewed from a distance in a manner of bird's eye viewing is included inthe monitor game image. The CPU 10 sets positions of the first virtualcamera and the second virtual camera in the virtual world (parametersrelating to the first virtual camera and the second virtual camera),based on the position of the player object Po represented by theposition data Dai, and the operation instruction direction (thedirection of the player object Po) in accordance with the operationinstruction direction data Da12. In this way, the terminal game imageand the monitor game image are game images of the same virtual worldviewed from different viewpoints. Therefore, the game images of thevirtual world viewed from the different viewpoints are displayed on theLCD 61 and the monitor 2, respectively. Although the direction of thefirst virtual camera is set based on the operation instruction direction(the direction of the player object Po), since the operation instructiondirection (the direction of the player object Po) is set based on theattitude of the terminal device 6 as described above, the direction ofthe first virtual camera is eventually controlled based on the attitudeof the terminal device 6. That is, the user is allowed to control thefirst virtual camera for generating the virtual world image displayed onthe LCD 61 of the terminal device 6, by moving the body of the terminaldevice 6. Since the attitude and the direction of the terminal device 6are associated with the attitude and the direction of the first virtualcamera, the user is allowed to experience a sense of looking into thevirtual world through the LCD 61 of the terminal device 6.

Next, the CPU 10 sets a movement trajectory of the player object Po(step 85), and ends the process of the sub-routine. For example, the CPU10 adds the current position of the player object Po, which isrepresented by the position data Dai, to the movement trajectoryrepresented by the movement trajectory data Dak, and updates themovement trajectory data Dak by using the movement trajectory to whichthe current position is added.

Referring back to FIG. 12, after the game control process at step 44,the CPU 10 and the GPU 32 generate a monitor game image to be displayedon the monitor 2 (step 45), and proceed to the next step. For example,the CPU 10 and the GPU 32 read, from the main memory, the datarepresenting the result of the game control process performed at step44, and read, from the VRAM 34, the data required for generating amonitor game image. Then, the CPU 10 and the GPU 32 generate a gameimage by using the read data, and store the generated monitor game imagein the VRAM 34. Any monitor game image may be generated by any method solong as the monitor game image represents the result of the game controlprocess performed at step 44. For example, the monitor game image may bea three-dimensional CG image which is generated through the steps of:arranging the second virtual camera in the virtual world based on theparameters relating to the second virtual camera, which are representedby the virtual camera data Daj; arranging the player object Po in thevirtual world based on the action/attitude data Dag and the positiondata Dai; arranging the movement trajectory Lp in the virtual worldbased on the movement trajectory data Dak; and calculating the scene ofthe virtual world viewed from the second virtual camera.

Next, the CPU 10 and the GPU 32 generate a terminal game image to bedisplayed on the terminal device 6 (step 46), and proceed to the nextstep. For example, the CPU 10 and the GPU 32 read, from the main memory,the data representing the result of the game control process performedat step 44, and read, from the VRAM 34, the data required for generatinga terminal game image. Then, the CPU 10 and the GPU 32 generate aterminal game image by using the read data, and store the generatedterminal game image in the VRAM 34. Like the monitor game image, anyterminal game image may be generated by any method so long as theterminal game image represents the result of the game control processperformed at step 44. The terminal game image may be generated by thesame method as, or a different method from, that for the monitor gameimage. For example, the terminal game image may be a three-dimensionalCG image which is generated through the steps of: arranging the firstvirtual camera in the virtual world based on the parameters relating tothe first virtual camera, which are represented by the virtual cameradata Daj; arranging the player object Po in the virtual world based onthe action/attitude data Dag and the position data Dai; and calculatingthe scene of the virtual world viewed from the first virtual camera.

When the action of the player object Po represented by theaction/attitude data Dag is an action of doing a kick (action of doing adolphin kick or a flutter kick), the player object Po may be arranged inthe virtual world so as to perform the action of doing a kick. Thereby,it is possible to cause the player object Po to move in synchronizationwith the action (stepping or knee bending) the user performs on theboard type controller 9.

Next, the CPU 10 generates a monitor game sound to be output to theloudspeakers 2 a of the monitor 2 (step 47), and proceeds to the nextstep. For example, the CPU 10 causes the DSP 33 to generate a monitorgame sound to be output from the loudspeakers 2 a, in accordance withthe result of the game control process performed at step 44. In anexample, the CPU 10 causes the DSP 33 to generate a monitor game soundin which BGM or the like to be output from the monitor 2 is added tovoices and action sounds of the objects, sound effects and the like,which are supposed to be heard on the basis of the position of thesecond virtual camera in the virtual world which is set based on theresult of the game control process at step 44.

Next, the CPU 10 generates a terminal game sound to be output to theloudspeakers 607 of the terminal device 6 (step 48), and proceeds to thenext step. For example, the CPU 10 causes the DSP 33 to generate aterminal game sound to be output from the loudspeakers 607, inaccordance with the result of the game control process performed at step44. In an example, the CPU 10 causes the DSP 33 to generate a terminalgame sound in which BGM or the like to be output from the terminaldevice 6 is added to voices and action sounds of the objects, soundeffects and the like, which are supposed to be heard on the basis of theposition of the first virtual camera in the virtual world which is setbased on the result of the game control process at step 44. The terminalgame sound may be the same as or different from the monitor game sound.Further, the terminal game sound may be partially different from themonitor game sound. For example, the terminal game sound and the monitorgame sound may include the same BGM and different sound effects. Whenthe monitor game sound and the terminal game sound are the same, theterminal game sound generation step at step 48 need not be performed.

Next, the CPU 10 outputs the monitor game image and the monitor gamesound to the monitor 2 (step 49), and proceeds to the next step. Forexample, the CPU 10 transmits, to the AV-IC 15, the data of the monitorgame image stored in the VRAM 34 and the data of the monitor game soundgenerated by the DSP 33. The AV-IC 15 transmits the data of the monitorgame image and the data of the monitor game sound through the AVconnector 16 to the monitor 2. Thereby, the monitor game image isdisplayed on the monitor 2, and the monitor game sound is output fromthe loudspeakers 2 a.

Next, the CPU 10 transmits the terminal game image and the terminal gamesound to the terminal device 6 (step 50), and proceeds to the next step.For example, the CPU 10 transmits, to the codec LSI 27, the data of theterminal game image stored in the VRAM 34 and the data of the terminalgame sound generated by the DSP 33. The codec LSI 27 performs apredetermined compression process on these data. The compressed data ofthe terminal game image and the compressed data of the terminal gamesound are transmitted from the codec LSI 27 to the terminalcommunication module 28, and then transmitted from the terminalcommunication module 28 to the terminal device 6 via the antenna 29. Thedata of the terminal game image and the data of the terminal game soundtransmitted from the game apparatus body 5 are received by the wirelessmodule 610 of the terminal device 6, and are subjected to apredetermined decompression process by the codec LSI 606. Thedecompressed data of the terminal game image is output to the LCD 61,and the decompressed data of the terminal game sound is output to thesound IC 608. Thereby, the terminal game image is displayed on the LCD61, and the terminal game sound is output from the loudspeakers 607.

Next, the CPU 10 determines whether to end the game (step 51). The gameis to be ended, for example, when conditions for game over or game clearare satisfied, or when the user has performed an operation to end thegame. When the game is not to be ended, the CPU 10 returns to step 42and repeats the processing, whereas when the game is to be ended, theCPU 10 ends the processing of the flowchart. Thereafter, the series ofprocess steps 42 to 51 are repeatedly executed until the CPU 10determines to end the game at step 51.

According to the above-mentioned processing, when the user performs anoperation based on his/her action on the board type controller 9 andthereby a process based on the user's action is performed, the user isallowed to view the result of the process on the LCD 61 of the portableterminal device 6 held by the user. Therefore, the user, who performs anaction with the terminal device 6 in his/her hands, is allowed to viewthe result of the process in a favorable situation. Further, the user isallowed to control the virtual camera, and the attitude and the movementdirection of the player object Po by moving the portable terminal device6. Therefore, it is possible to provide the user with a sense ofpresence in the virtual world, and with an operation sense of becomingthe player object Po.

Second Embodiment

The game played by using the terminal device 6 and the board typecontroller 9 is not limited to the above-mentioned first exemplary game,but various exemplary games are considered. In the following descriptionof the outline of information processing according to a secondembodiment, a second exemplary game is adopted as an example of theinformation processing. Since an information processing apparatus forexecuting an information processing program according to the secondembodiment, and an information processing system including theinformation processing apparatus are identical to those of the firstembodiment, the same components as those of the first embodiment aredesignated by the same reference numerals, and the detailed descriptionthereof is omitted.

Firstly, an outline of information processing performed on the gameapparatus body 5 will be described with reference to FIGS. 18A, 18B, 19,and 20 before a specific description of process steps performed by thegame apparatus body 5 is given. FIG. 18A shows an example of an imagedisplayed on the LCD 61 of the terminal device 6 in the second exemplarygame. FIG. 18B shows an example of an image displayed on the monitor 2in the second exemplary game. FIG. 19 shows another example of an imagedisplayed on the LCD 61 of the terminal device 6 in the second exemplarygame. FIG. 20 is a schematic diagram illustrating an example in whichthe position and the attitude of a virtual camera vary depending on theattitude of the terminal device 6.

In the second exemplary game, as in the first exemplary game, a userperforms an operation by using the terminal device 6 and the board typecontroller 9. Specifically, as in the first exemplary game, the userstands on the board type controller 9 with the terminal device 6 inhis/her hands, and performs an operation. Then, the user performs anaction on the board type controller 9 (e.g., the user performs kneebending on the board type controller 9, or adjusts the center-of-gravityposition of a load applied to the board type controller 9), andsimultaneously performs an operation to move the terminal device 6,while viewing the image displayed on the monitor 2 and the imagedisplayed on the LCD 61 of the terminal device 6, thereby playing thegame. On each of the LCD 61 of the terminal device 6 and the monitor 2,a game image is displayed, in which a player object Po and a virtualcamera in a virtual world each perform an action (e.g., an action of theplayer object Po to change its attitude and movement direction, and anaction of the virtual camera to change its position and attitude) inaccordance with the direction and the attitude of the terminal deviceheld by the user, and the action of the user on the board typecontroller 9, and furthermore, the position and the attitude of thevirtual camera set in the virtual world are changed in accordance withthe position and the attitude of the player object Po and/or the actionof the user.

As shown in FIG. 18A, on the LCD 61 of the terminal device 6, a playerobject Po moving in a virtual world is displayed. In the example shownin FIG. 18A, a virtual camera is located near the back of the playerobject Po which rides on a pogo stick on the ground in the virtualworld, and a state of the virtual world viewed from the virtual camerais displayed together with the player object Po. Displaying, on the LCD61, the virtual world viewed at close range from behind the playerobject Po allows the user holding the terminal device 6 to experience asense of presence in the virtual world, and to intuitively know themovement direction and the movement speed of the player object Po.

Further, as shown in FIG. 18B, the same virtual world as that displayedon the LCD 61 is displayed on the monitor 2. In the example shown inFIG. 18B, a state of the virtual world in which the player object Po onthe pogo stick is viewed from a distance in a manner of bird's eyeviewing, is displayed together with the player object Po. Displaying, onthe monitor 2, the state of the virtual world in which the player objectPo is viewed from a distance allows the user to easily know thesurroundings of the player object Po, and allows another person whowatches the user's playing the second exemplary game to enjoy a view inwhich the player object Po moves in the virtual world.

For example, when the user performs an action such as knee bending(jumping) on the board type controller 9, the player object Po jumpswith the pogo stick at a speed according to the knee bending. When theuser performs an action to shift the center-of-gravity position of theload applied to the board type controller 9 while the player object Pois in the air in the virtual world, the movement direction of the playerobject Po in the air is changed to a direction according to the shift ofthe center-of-gravity position. In this way, the user is allowed tochange the jumping power (movement speed) and the movement direction ofthe player object Po in accordance with his/her action on the board typecontroller 9.

When the user changes the attitude of the terminal device 6 or rotatesthe terminal device 6 to change its direction, the movement direction inwhich the player object Po jumps out, or the position and the attitudeof the virtual camera is changed. For example, when the user turns theterminal device 6 to the left or to the right in the real space, thedirection in which the player object Po moves in the virtual worldchanges in accordance with the direction change of the terminal device6. Further, in accordance with the change in the movement direction, theposition and the attitude of the virtual camera change such that ahorizontal line-of-sight direction obtained by projecting theline-of-sight direction of the virtual camera on the horizontal plane inthe virtual world becomes equal to a horizontal movement directionobtained by projecting the movement direction on the horizontal plane inthe virtual world. Specifically, when the user changes the direction ofthe terminal device 6 such that the back surface thereof turns to theleft, the movement direction of the player object Po changes to theleftward direction in the virtual world. Then, the line of sight of thevirtual camera turns to the movement direction, and the position and theattitude of the virtual camera change such that the virtual camera islocated near and behind the player object Po.

When the user turns the terminal device 6 upward or downward in the realspace, the position and the attitude of the virtual camera change inaccordance with the direction change of the terminal device 6. Forexample, as shown in FIG. 20, when the user holds the terminal device 6in such an attitude that the back surface of the terminal device 6 facesthe horizontal direction in the real space (i.e., the back surface ofthe terminal device 6 or the display surface of the LCD 61 is orthogonalto the horizontal direction), the virtual camera is located in asubjective position near the back of the player object Po in such anattitude that the line-of-sight direction of the virtual camera is equalto the horizontal direction in the virtual world. In this case, avirtual world as shown in FIG. 18A is displayed on the LCD 61 of theterminal device 6. On the other hand, when the user holds the terminaldevice 6 in such an attitude that the back surface of the terminaldevice 6 faces downward in the real space (i.e., the back surface of theterminal device 6 or the display surface of the LCD 61 is orthogonal tothe vertical direction, and the back surface faces downward), thevirtual camera is located in a position (overhead position) in which theline of sight of the virtual camera is equal to the vertical directionin the virtual world and the virtual camera views the player object Pofrom above, and the point of view is in front of the player object Po inthe movement direction. In this case, a virtual world as shown in FIG.19 is displayed on the LCD 61 of the terminal device 6. When the userchanges the direction of the terminal device 6 from the horizontaldirection to the downward direction in the real space, the attitude andthe position of the virtual camera also change from the subjectiveposition to the overhead position in accordance with the attitude of theterminal device 6.

In this way, by associating the attitude and the direction of theterminal device 6 with the attitude and the direction of the virtualcamera, the user is allowed to experience a sense of looking into thevirtual world through the LCD 61 of the terminal device 6. Further, whenthe user holds the terminal device 6 in such an attitude that the backsurface of the terminal device 6 faces downward in the real space, thefeet of the player object Po are displayed on the LCD 61 of the terminaldevice 6, which allows the user to check the landing point of the playerobject Po or surroundings of the feet of the player object Po. That is,when the user changes the up-down direction of the terminal device 6,the user is allowed to secure a field of view in the movement directionof the player object Po, or a field of view toward the feet of theplayer object Po, and furthermore, allowed to selectively displayhis/her desired image on the LCD 61 in accordance with the gamesituation.

Next, the process of the second embodiment, which is performed by thegame system 1, will be described in detail. Firstly, main data used inthe process according to the second embodiment will be described withreference to FIG. 21. FIG. 21 is a diagram illustrating an example ofmain data and programs stored in the main memory of the game apparatusbody 5 of the second embodiment.

As shown in FIG. 21, in a data storage area of the main memory, boardoperation data Dba, terminal operation data Dbb, load value data Dbc,center-of-gravity position data Dbd, terminal device direction/attitudedata Dbe, operation direction data Dbf, action/attitude data Dbg,movement vector data Dbh, position data Dbi, virtual camera data Dbj,jumping power data Dbk, jump flag data Dbm, squat flag data Dbn, imagedata Dbo, and the like are stored. In addition to the data shown in FIG.21, data required for a game process, such as image data of variousobjects displayed on the monitor 2 and the LCD 61, and sound data usedfor a game, are optionally stored in the main memory. In a programstorage area of the main memory, various programs Pb constituting aninformation processing program of the second embodiment are stored.

The board operation data Dba includes load data Dba1 and the like. Sincethe load data Dba1 is identical to the load data Daa1 of the firstembodiment, repeated description is not necessary.

The terminal operation data Dbb includes acceleration data Dbb1, angularvelocity data Dbb2, and the like. Since the acceleration data Dbb1 andthe angular velocity data Dbb2 are identical to the acceleration dataDab1 and the angular velocity data Dab2 of the first embodiment,respectively, repeated description is not necessary.

Since the load value data Dbc and the center-of-gravity position dataDbd are identical to the load value data Dac and the center-of-gravityposition data Dad of the first embodiment, respectively, repeateddescription is not necessary.

The terminal device direction/attitude data Dbe includes real spacereference direction data Dbe1, current direction data Dbe2, and thelike. Since the real space reference direction data Dbe1 and the currentdirection data Dbe2 are identical to the real space reference directiondata Dae1 and the current direction data Dae2 of the first embodiment,respectively, repeated description is not necessary.

The operation direction data Dbf includes virtual world referencedirection data Dbf1, operation instruction direction data Dbf2, and thelike. The virtual world reference direction data Dbf1 is datarepresenting a virtual world reference direction set in the virtualworld. The operation instruction direction data Dbf2 is datarepresenting an operation instruction direction which is currentlyinstructed in the virtual world by the user. Methods for calculating thevirtual world reference direction and the operation instructiondirection of the second embodiment will be described later.

The action/attitude data Dbg, the movement vector data Dbh, the positiondata Dbi, and the virtual camera data Dbj are, as in the firstembodiment, data representing the action and the attitude of the playerobject Po, data representing the movement speed and the movementdirection of the player object Po, data representing the position of theplayer object Po, and data relating to the virtual camera, respectively,although the calculation methods thereof are different from those of thefirst embodiment as described later.

The jumping power data Dbk is data representing a jumping power(movement speed) of the player object Po, which is calculated based onthe user's action on the board type controller 9.

The jump flag data Dbm is data representing a jump flag which is turnedon while the player object Po is jumping. The squat flag data Dbn isdata representing a squat flag which is turned on when it is determinedthat the user squats on the board type controller 9.

The image data Dbo includes player object data Dbo1, background imagedata Dbo2, and the like. The player object data Dbo1 is data forarranging the player object Po of the second exemplary game in thevirtual world to generate a game image. The background image data Dbo2is data for arranging a background of the second exemplary game in thevirtual world to generate a game image.

Next, the process of the second embodiment, which is performed by thegame apparatus body 5, will be described in detail with reference toFIGS. 22 to 24. FIG. 22 is a sub-routine illustrating an example of aplayer object setting process of the second embodiment executed by thegame apparatus body 5. FIG. 23 is a sub-routine illustrating an exampleof a jumping operation setting process at step 165 in FIG. 22. FIG. 24is a diagram illustrating an example of “squat” determination and“stretch” determination with respect to variation in the total loadvalue. In the flowcharts of FIGS. 22 and 23, process steps for causingthe player object Po to move and displaying the same, in accordance withan operation performed by a user using the terminal device 6 and theboard type controller 9, will be mainly described among the wholeprocess, while detailed description of other process steps is not given.Further, in FIGS. 22 to 23, each of the steps executed by the CPU 10 isabbreviated as “S”.

When the game apparatus body 5 is powered on, the CPU 10 of the gameapparatus body 5 executes a boot program stored in the ROM/RTC 13 toinitialize each unit such as the main memory. Then, the informationprocessing program stored in the optical disc 4 is loaded to the mainmemory, and the CPU 10 starts to execute the program. The process of thesecond embodiment to be performed after completion of theabove-mentioned process steps is identical to the process of the firstembodiment except the game control process at step 44 in the main flowshown in FIG. 12. Therefore, the identical part of the process is notrepeatedly described. Further, the game control process at step 44according to the second embodiment is identical to the sub-routine ofFIG. 13 of the first embodiment except the player object setting processat step 83, and except that the movement trajectory setting process atstep 85 is not performed. Therefore, the identical part of the processis not repeatedly described. Hereinafter, the player object settingprocess at step 83 according to the second embodiment will be describedwith reference to FIG. 22.

In FIG. 22, the CPU 10 calculates a horizontal angular differencebetween the real space reference direction and the current direction(step 161), and proceeds to the next step. The horizontal angulardifference is an angular difference which is obtained by projecting anangular difference between the real space reference direction and thecurrent direction in the real space on the horizontal plane, as in thefirst embodiment. The horizontal angular difference indicates an angleat which the direction of the terminal device 6 (the direction in whichthe back surface of the terminal device 6 faces (the z-axis positivedirection shown in FIG. 3)) is changed from the initial attitude of theterminal device 6 around vertical direction in the real space. Forexample, the CPU 10 calculates a horizontal angular difference by usingthe real space reference direction represented by the real spacereference direction data Dbe1, and the current direction represented bythe current direction data Dbe2.

Next, the CPU 10 calculates an operation instruction direction withrespect to the virtual world reference direction, in accordance with thehorizontal angular difference and the attitude (up/down angle) of theterminal device 6 in the up-down direction (step 162), and proceeds tothe next step. For example, the CPU 10 calculates, based on the virtualworld reference direction represented by the virtual world referencedirection data Dbf1, an operation instruction direction in the virtualworld such that an angular difference which occurs when the virtualworld reference direction and the operation instruction direction areprojected on the horizontal plane in the virtual world is equal to thehorizontal angular difference, and the same positional relation isachieved between the real space and the virtual world (i.e., if thecurrent direction rotates to the left with respect to the real spacereference direction, the operation instruction direction also rotates tothe left with respect to the virtual world reference direction).Further, the CPU 10 calculates, based on the current directionrepresented by the current direction data Dbe2, an operation instructiondirection in the virtual world such that an angular difference betweenthe horizontal direction and the operation instruction direction in thevirtual world is equal to an angular difference between the horizontaldirection and the current direction in the real space, and the samepositional relation is achieved between the real space and the virtualworld (that is, if the current direction is the downward direction withrespect to the horizontal direction in the real space, the operationinstruction direction is also the downward direction with respect to thehorizontal direction in virtual world). Then, the CPU 10 updates theoperation instruction direction data Dbf2 by using the calculatedoperation instruction direction.

In the setting of the operation instruction direction at step 162, arange in which the operation instruction direction can be set may belimited. For example, the operation instruction direction may be setwithin a predetermined angular range on the left side and the right sideof the virtual world reference direction with respect to the horizontaldirection in the virtual world, and within a predetermined angular rangeon the upper side and the lower side of the horizontal direction withrespect to the vertical direction in the virtual world. In this case, ifthe calculated operation instruction direction is outside theabove-mentioned angular range, the operation instruction direction isshifted into the angular range closest to the calculated operationinstruction direction. In another example, a current direction may beset within a predetermined angular range on the left side and the rightside of the real space reference direction with respect to thehorizontal direction in the real space, and within a predeterminedangular range on the upper side and the lower side of the horizontaldirection with respect to the vertical direction in the real space. Ifthe calculated current direction is outside the above-mentioned angularrange, the current direction is shifted into the angular range closestto the calculated current direction. Also in this case, since theoperation instruction direction is calculated based on the currentdirection limited within the angular range, the operation instructiondirection is also set within the angular ranges set in the upward,downward, leftward, or rightward direction.

Next, the CPU 10 determines whether or not the jump flag is set to OFF(step 163). For example, when the jump flag represented by the jump flagdata Dbm is set to OFF, the CPU 10 determines that the player object Pois not jumping, and proceeds to step 164. On the other hand, when thejump flag represented by the jump flag data Dbm is set to ON, the CPU 10determines that the player object Po is jumping, and proceeds to step166.

In step 164, the CPU 10 sets the object action in a pre jumping state,and proceeds to next step. For example, the CPU 10 sets the action ofthe player object Po in a pre jumping state (e.g., a state in which theplayer object Po lands on the ground in the virtual world and tries totake a next jump), and updates the action/attitude data Dbg by using theset action.

Next, the CPU 10 performs a jumping action setting process (step 165),and ends the process of this sub-routine. Hereinafter, the jumpingaction setting process performed at step 165 will be described withreference to FIG. 23.

In FIG. 23, the CPU 10 determines whether or not the squat flagrepresented by the squat flag data Dbn is set to OFF (step 181). Whenthe squat flag is set to OFF, the CPU 10 proceeds to step 182. On theother hand, when the squat flag is set to ON, the CPU 10 proceeds tostep 185.

In step 182, the CPU 10 determines whether or not the total load valueis equal to or greater than a first threshold. For example, the CPU 10determines whether or not the latest total load value represented by theload value data Dbc is equal to or greater than the first threshold. Thefirst threshold is a threshold value for determining that the userperforms an action of squatting down on the board type controller 9. Forexample, the first threshold is set at a value obtained by adding apredetermined rate (e.g., 20%) to the weight of the user on the boardtype controller 9. The weight of the user on the board type controller 9may be calculated based on the total load value (e.g., an average oftotal load values obtained in a predetermined period), or may bepreviously inputted by the user. When the total load value is equal toor greater than the first threshold, the CPU 10 proceeds to step 183. Onthe other hand, when the total load value is smaller than the firstthreshold, the CPU 10 proceeds to step 185.

When the user performs an action of bending and stretching (jumping) onthe board type controller 9, it is general that the user firstly takesan action of squatting on the board type controller 9 and then takes anaction of stretching upward. As shown in FIG. 24, when the user takes anaction of squatting on the board type controller 9, the total load valueonce falls below the weight of the user and thereafter exceeds theweight. That is, the CPU 10 can detect that the user has performed anaction of squatting, by detecting that the total load value exceeds theweight of the user (the total load value becomes equal to or greaterthan the first threshold).

In step 183, the CPU 10 sets the object action to the squatting action,and proceeds to the next step. For example, the CPU 10 sets the actionof the player object Po to the squatting action (e.g., a state in whichthe player object Po squats on the pogo stick for a next jump, with thepogo stick landing on the ground in the virtual world), and updates theaction/attitude data Dbg by using the set action.

Next, the CPU 10 sets the squat flag to ON (step 184), and proceeds tostep 185. For example, the CPU 10 sets the squat flag represented by thesquat flag data Dbn to ON.

In step 185, the CPU 10 determines whether or not the squat flagrepresented by the squat flag data Dbn is set to ON. When the squat flagis set to ON, the CPU 10 proceeds to step 186. On the other hand, whenthe squat flag is set to OFF, the CPU 10 ends the process of thissub-routine.

In step 186, the CPU 10 determines whether or not the total load valueis smaller than a second threshold. For example, the CPU 10 determineswhether or not the latest total load value represented by the load valuedata Dbc is smaller than the second threshold. The second threshold is athreshold value for determining that the user performs an action ofstretching on the board type controller 9. For example, the secondthreshold is set at a value obtained by subtracting a predetermined rate(e.g., 20%) from the weight of the user on the board type controller 9.When the total load value is equal to or greater than the secondthreshold, the CPU 10 proceeds to step 187. On the other hand, when thetotal load value is smaller than the second threshold, the CPU 10proceeds to step 188.

When the user performs an action of bending and stretching (jumping) onthe board type controller 9, it is general that the user firstly takesan action of squatting on the board type controller 9 and then takes anaction of stretching upward. As shown in FIG. 24, when the user takes anaction of stretching on the board type controller 9, the total loadvalue once exceeds the weight of the user and thereafter falls below theweight. That is, the CPU 10 can detect that the user has performed anaction of stretching, by detecting that the total load value falls belowthe weight of the user (the total load value becomes smaller than thesecond threshold) after the detection of the squatting action.

In step 187, the CPU 10 adds a jumping power, and ends the process ofthis sub-routine. For example, the CPU 10 adds a predetermined amount tothe parameter of the jumping power represented by the jumping power dataDbk, and updates the jumping power data Dbk by using the parameter ofthe jumping power after the addition. The process at step 187 isperformed when the determination result is positive (i.e., the squatflag is ON) at step 185, and the determination result is negative (i.e.,the total load value is equal to or greater than the second threshold)at step 186. That is, the process at step 187 is performed during aperiod from when the user squats to when it is determined that the userstretches. The longer the period (“squat” period) is, the more thejumping power is increased gradually.

On the other hand, in step 188, the CPU 10 sets the object action to thestretching action, and proceeds to the next step. For example, the CPU10 sets the action of the player object Po to the stretching action(e.g., an action of jumping into the air with the pogo stick from thestate in which the pogo stick lands on the ground in the virtual world),and updates the action/attitude data Dbg by using the set action.

Next, the CPU 10 sets a movement vector of the player object Po, basedon the operation instruction direction and the jumping power (step 189),and proceeds to the next step. For example, the CPU 10 calculates amovement speed of the player object Po, based on the parameter of thejumping power represented by the jumping power data Dbk. In an example,the CPU 10 calculates a movement speed of the player object Po so as tobe proportional to the parameter of the jumping power. Further, the CPU10 sets a movement direction of the player object Po, based on theoperation instruction direction represented by the operation instructiondirection data Dbf2. For example, a movement direction is set such thata horizontal movement direction obtained by projecting the movementdirection on the horizontal plane in the virtual world is equal to ahorizontal operation instruction direction obtained by projecting theoperation instruction direction on the horizontal plane in the virtualworld. Further, a climb angle in the movement direction, at which theplayer object Po moves upward in the virtual world, is set to a specificangle. Then, the CPU 10 calculates a movement vector in the virtualworld by using the movement speed and the movement direction of theplayer object Po, and updates the movement vector data Dbh by using thecalculated movement vector. For example, the CPU 10 sets a length of themovement vector by using the calculated movement speed of the playerobject Po, and sets a direction of the movement vector based on theoperation instruction direction represented by the operation instructiondirection data Dbf2.

Next, the CPU 10 initializes the jumping power (step 190), and proceedsto the next step. For example, the CPU 10 initializes the parameter ofthe jumping power represented by the jumping power data Dbk to apredetermined initial value, thereby to update the jumping power dataDbk. The initial value at step 190 may be a value indicating that thejumping power is 0, or may be a value indicating a relatively lowjumping power.

Next, the CPU 10 sets the squat flag to OFF (step 191), and proceeds tothe next step. For example, the CPU 10 sets the squat flag representedby the squat flag data Dbn to OFF.

Next, the CPU 10 sets the jump flag to ON (step 192), and ends theprocess of this sub-routine. For example, the CPU 10 sets the jump flagrepresented by the jump flag data Dbm to ON.

Referring back to FIG. 22, when it is determined at step 163 that thejump flag is ON, the CPU 10 sets the object action to “during jumping”(step 166), and proceeds to the next step. For example, the CPU 10 setsthe action of the player object Po to “during jumping” (e.g., a state inwhich the player object Po on the pogo stick moves in the air in thevirtual world), and updates the action/attitude data Dbg by using theset action.

Next, the CPU 10 corrects the movement vector of the player object Pobased on the center-of-gravity position (step 167), and proceeds to thenext step. For example, the CPU 10 corrects the movement vectorrepresented by the movement vector data Dbh by using the latestcenter-of-gravity position represented by the center-of-gravity positiondata Dbd, and updates the movement vector data Dbh by using thecorrected movement vector. In an example, the CPU 10 calculates acorrection vector extending from the center of the platform 9 a to thecenter-of-gravity position, and adds the correction vector to themovement vector that is currently set to correct the movement vector.The length of the correction vector may be a value obtained bymultiplexing the distance between the center of the platform 9 a and thecenter-of-gravity position by a predetermined constant, or may be afixed value. For example, if the center-of-gravity position is presenton a position shifted by a distance D from the center of the platform 9a to the left, a correction vector having a length according to thedistance D in the leftward direction in the virtual world is calculated,and added to the currently set movement vector. Thereby, the movementvector is corrected toward the position where the center-of-gravityposition is present by an amount according to the distance between thecenter of the platform 9 a and the center-of-gravity position.

Next, the CPU 10 corrects the movement vector of the player object Pobased on the environment set in the virtual world (step 168), andproceeds to the next step. For example, the CPU 10 corrects the movementvector represented by the movement vector data Dbh, based on theenvironment (gravity, wind, influence of other objects, and the like)set in the virtual world, and updates the movement vector data Dbh byusing the corrected movement vector.

Next, the CPU 10 calculates a position of the player object Po (step169), and proceeds to the next step. For example, the CPU 10 causes theposition of the player object Po represented by the position data Dbi toshift in the virtual world, based on the movement vector represented bythe movement vector data Dbh, and updates the position data Dbi by usingthe position after the shifting.

Next, the CPU 10 determines whether or not the player object Po haslanded from the air in the virtual world (step 170). For example, theCPU 10 determines whether or not the player object Po has landed (e.g.,the position of the player object Po is on a topography object oranother object in the virtual world) based on the position of the playerobject Po represented by the position data Dbi. When the result of thedetermination is that the player object Po has landed, the CPU 10proceeds to step 171. On the other hand, when the result of thedetermination is that the player object Po has not landed, the CPU 10ends the process of this sub-routine.

In step 171, the CPU 10 initializes the movement vector of the playerobject Po, and proceeds to the next step. For example, the CPU 10initializes the movement vector represented by the movement vector dataDbh to update the movement vector data Dbh. In step 171, the movementvector may be initialized to a parameter indicating that the playerobject Po does not move, or may be initialized to a predeterminedmovement vector.

Next, the CPU 10 sets the jump flag to OFF (step 172), and ends theprocess of this sub-routine. For example, the CPU 10 sets the jump flagrepresented by the jump flag data Dbm to OFF.

In the virtual camera setting process (step 84) of the secondembodiment, a first virtual camera for generating a terminal game imageis set such that a first player object Po1 arranged in the virtual worldis included in the display range, based on the operation instructiondirection represented by the operation instruction direction data Dbf2.Specifically, as described with reference to FIG. 20, when the operationinstruction direction is the horizontal direction in the virtual world,the operation instruction direction is regarded as the line-of-sightdirection, and the player object Po is arranged substantially in thecenter of the view volume, and the first virtual camera is set at thesubjective position near the player object Po. When the operationinstruction direction is the vertical direction in the virtual world,the operation instruction direction is regarded as the line-of-sightdirection, and a position in front of the player object Po in themovement direction of the player object Po is regarded as a point ofview, and the first virtual camera is set at the overhead position atwhich the first virtual camera views the player object Po from above.When the operation instruction direction is between the horizontaldirection and the vertical direction, the attitude and the position ofthe first virtual camera are also changed from the subjective positionto the overhead position in accordance with the direction of theoperation instruction direction. On the other hand, a second virtualcamera for generating a monitor game image is set in the same virtualworld as the first virtual camera such that a state of the virtual worldin which the player object Po is viewed from a distance in a manner ofbird's eye viewing is included in the monitor game image. The CPU 10sets the positions of the first virtual camera and the second virtualcamera in the virtual world (the parameters of the first virtual cameraand the second virtual camera), based on the position of the playerobject Po represented by the position data Dbi and the operationinstruction direction represented by the operation instruction directiondata Dbf2.

In the monitor game image generation process (step 45) and the terminalgame image generation process (step 46) of the second embodiment, imagesrepresenting the result of the game control process are generated,respectively, as in the first embodiment. For example, the monitor gameimage is generated as a three-dimensional CG image which is generatedthrough the steps of: arranging the second virtual camera in the virtualworld based on the parameters relating to the second virtual camera,which are represented by the virtual camera data Dbj; arranging theplayer object Po and the like in the virtual world based on theaction/attitude data Dbg and the position data Dbi; and calculating ascene of the virtual world viewed from the second virtual camera. Theterminal game image is generated as a three-dimensional CG image whichis generated through the steps of: arranging the first virtual camera inthe virtual world based on the parameters relating to the first virtualcamera, which are represented by the virtual camera data Dbj; arrangingthe player object Po and the like in the virtual world based on theaction/attitude data Dbg and the position data Dbi; and calculating ascene of the virtual world viewed from the first virtual camera.

In this way, according to the above-mentioned process, when the userperforms an operation based on his/her action on the board typecontroller 9 and thereby a process based on the user's action isperformed, the user is allowed to view the result of the process on theLCD 61 of the portable terminal device 6 held by the user. Therefore,the user, who performs an action with the terminal device 6 in his/herhands, is allowed to view the result of the process in a favorablesituation for the user. Further, since the user is allowed to controlthe position and the attitude of the virtual camera or the attitude andthe movement direction of the player object Po by moving the portableterminal device 6, it is possible to provide the user with a sense ofpresence in the virtual world, and with an operation sense of becomingthe player object Po. Furthermore, in the second exemplary game, themovement direction of the player object Po is determined based on theattitude of the portable terminal device 6, but the movement directionafter the player object Po has jumped is adjusted in accordance with theaction of the user on the board type controller 9. Accordingly, themovement direction of the player object Po can be changed not only by anoperation using the terminal device 6 but also by an operation using theboard type controller 9, which provides the user with a new operationsense.

Third Embodiment

Hereinafter, information processing according to a third embodiment willbe described based on another exemplary game which is played by usingthe terminal device 6 and the board type controller 9. In the followingdescription of the outline of the information processing according tothe third embodiment, a third exemplary game is adopted as an example ofthe information processing. Since an information processing apparatusfor executing an information processing program according to the thirdembodiment, and an information processing system including theinformation processing apparatus are identical to those of the firstembodiment, the same components as those of the first embodiment aredesignated by the same reference numerals, and the detailed descriptionthereof is omitted.

Firstly, an outline of information processing performed on the gameapparatus body 5 will be described with reference to FIGS. 25A, 25B, and26 before a specific description of processes of the third embodimentperformed by the game apparatus body 5 is given. In the followingdescription of the outline of the information processing according tothe third embodiment, a third exemplary game is adopted as an example ofthe information processing. FIG. 25A shows an example of an imagedisplayed on the LCD 61 of the terminal device 6 in the third exemplarygame. FIG. 25B shows an example of an image displayed on the monitor 2in the third exemplary game. FIG. 26 is a schematic diagram illustratingan example of actions of a first player object Po1 and a second playerobject Po2 in accordance with the attitude of the terminal device 6.

In the third exemplary game, as in the first exemplary game shown inFIG. 9, a user performs an operation by using the terminal device 6 andthe board type controller 9. Specifically, as in the first exemplarygame, the user stands on the board type controller 9 with the terminaldevice 6 in his/her hands, and performs an operation. Then, the userperforms an action on the board type controller 9 (e.g., an action ofplacing his/her weight forward and backward on the board type controller9), and performs an operation to move the terminal device 6, whileviewing the image displayed on the monitor 2 and the image displayed onthe LCD 61 of the terminal device 6, thereby playing the game. On theLCD 61 of the terminal device 6 and on the monitor 2, a game image isdisplayed, in which the first player object Po1, the second playerobject Po2, and a target object To each perform an action in a virtualworld (e.g., an action of the first player object Po1 to change itsattitude and movement direction, an action of the second player objectPo2 to change its direction, or an action of the target object To tomove) in accordance with the direction and the attitude of the terminaldevice held by the user, and the action of the user on the board typecontroller 9, and furthermore, the position and the attitude of avirtual camera set in the virtual world are changed in accordance withthe position and the attitude of the first player object Po1.

As shown in FIG. 25A, on the LCD 61 of the terminal device 6, a playerobject Po moving in a virtual world is displayed. In the example shownin FIG. 25A, the player object Po includes two player objects (a firstplayer object Po1 and a second player object Po2). As an example of theplayer object Po, a tank moving on the ground in the virtual world isdisplayed. Specifically, the first player object Po1 corresponds to thebody of the tank, and the second player object Po2 corresponds to aturret of the tank. A virtual camera is located near the back of thefirst player object Po, and a state of the virtual world viewed from thevirtual camera is displayed together with the player object Po. Further,in the example shown in FIG. 25A, the target object To, which is atarget of a shell shot by the tank, is arranged in the virtual world,and an expected trajectory S of the shell from a front end of the turret(second player object Po2) of the tank to the target object to isdisplayed. An enemy object Eo, which is an enemy of the player objectPo, is arranged in the virtual world. Displaying, on the LCD 61, thevirtual world viewed from near the back of the player object Po(specifically, the first player object Po1) allows the user holding theterminal device 6 to experience a sense of presence in the virtualworld, and to intuitively know the movement direction and the movementspeed of the player object Po.

Further, as shown in FIG. 25B, the same virtual world as that displayedon the LCD 61 is displayed on the monitor 2. In the example shown inFIG. 25B, a state of the virtual world in which the player object Po isviewed from a distance in a manner of bird's eye viewing, is displayedtogether with the player object Po, the target object To, the enemyobject Eo, and the expected trajectory S. Displaying, on the monitor 2,the state of the virtual world in which the player object Po is viewedfrom a distance allows the user to easily know the surroundings of theplayer object Po, and allows another person who watches the user'splaying the third exemplary game to enjoy a view in which the playerobject Po moves in the virtual world.

For example, when the user performs an action to shift forward thecenter-of-gravity position of a load the user applies to the board typecontroller 9, the player object Po moves forward in the direction of thefirst player object Po1 at a speed according to a forward length fromthe center of the platform 9 a to the center-of-gravity position. Whenthe user performs an action to shift backward the center-of-gravityposition of the load the user applies to the board type controller 9,the player object Po moves backward in the direction of the first playerobject Po1 at a speed according to a backward length from the center ofthe platform 9 a to the center-of-gravity position. In this way, theuser is allowed to move the player object Po forward and backward inaccordance with his/her action on the board type controller 9.

When the user changes the attitude of the terminal device 6 or rotatesthe terminal device 6 to change the direction of the terminal device 6,the direction of the player object Po and the position of the targetobject To change. Specifically, as shown in FIG. 26, when the directionof the terminal device 6 rotates within a turret/target directionoperation range corresponding to a turret/target movement angle range,the second player object Po2 (turret) rotates to the left or to theright in the virtual world, and the position of the target object To,which is a target of a shell shot from the turret, also moves in thevirtual world. When the direction of the terminal device 6 rotates intoa body direction operation range corresponding to a range outside theturret/target movement angle range, the first player object Pal (thetank body) as well as the second player object Po2 rotate to the left orto the right in the virtual world in accordance with the change in thedirection of the terminal device 6, and further, position of the targetobject To also shifts in the virtual world in accordance with therotation of the second player object Po2. Since the virtual camera isset near the back of the first player object Po1 as described above, theattitude and the position of the virtual camera change with the changein the direction of the first player object Po1. Accordingly, when theuser changes the left-right direction of the terminal device 6 withinthe turret/target direction operation range in the real space, the useris allowed to change the position of the target object To withoutchanging the position and the attitude of the virtual camera, andtherefore, the user is allowed to adjust the target position with thegame screen displayed on the LCD 61 being fixed. That is, the operationusing the terminal device 6 within the turret/target direction operationrange provides the user with favorable operation environment foradjusting the position of the target object To to attack the enemyobject Eo. Further, when the user changes the direction of the terminaldevice 6 to the left or to the right in the body direction operationrange in the real space, the user is allowed to change the direction ofthe first player object Po1 and the position and the direction of thevirtual camera. Therefore, the user is allowed to change the movementdirection of the player object Po, and adjusts the target position tothe outside of the game screen displayed on the LCD 61. In this way,setting at least two operation ranges allows the user to perform variousoperations by using the attitude of a single device.

When the direction of the terminal device 6 changes upward or downward,the direction of the turret (second player object Po2) changes upward ordownward in the virtual world with the change in the direction of theterminal device 6, and the position of the target object To, which is atarget of a shell shot from the turret, also changes in the virtualworld. That is, the user is allowed to change the direction of theturret (the position of the target object To) upward, downward,leftward, and rightward in accordance with the direction of the terminaldevice 6. Thus, the user is allowed to direct the turret (the arrangethe target object To) to his/her desired direction (position) by onlychanging the direction of the terminal device 6.

Next, the process of the third embodiment, which is performed by thegame system 1, will be described in detail. First, main data used in theprocess according to the third embodiment will be described withreference to FIG. 27. FIG. 27 is a diagram illustrating an example ofmain data and programs stored in the main memory of the game apparatusbody 5 of the third embodiment.

As shown in FIG. 27, in a data storage area of the main memory, boardoperation data Dca, terminal operation data Dcb, load value data Dcc,center-of-gravity position data Dcd, terminal device direction/attitudedata Dee, operation direction data Dcf, action/attitude data Dcg,movement vector data Dch, position data Dci, virtual camera data Dcj,image data Dck, and the like are stored. In addition to the data shownin FIG. 27, data required for a game process, such as image data ofvarious objects displayed on the monitor 2 and the LCD 61, and sounddata used for a game, are optionally stored in the main memory. In aprogram storage area of the main memory, various programs Pcconstituting an information processing program of the third embodimentare stored.

The board operation data Dca includes load data Dca1 and the like. Sincethe load data Dca1 is identical to the load data Daa1 of the firstembodiment, repeated description is not necessary.

The terminal operation data Dcb includes acceleration data Dcb1, angularvelocity data Dcb2, and the like. Since the acceleration data Dcb1 andthe angular velocity data Dcb2 are identical to the acceleration dataDab1 and the angular velocity data Dab2 of the first embodiment,respectively, repeated description is not necessary.

Since the load value data Dcc and the center-of-gravity position dataDcd are identical to the load value data Dac and the center-of-gravityposition data Dad of the first embodiment, respectively, repeateddescription is not necessary.

The terminal device direction/attitude data Dce includes real spacereference direction data Dce1, current direction data Dce2, and thelike. As in the first embodiment, the real space reference directiondata Dce1 is data representing the reference direction of the terminaldevice 6 in the real space, and the current direction data Dce2 is datarepresenting the direction and the attitude of the terminal device 6 atthe current time in the real space. However, in the third embodiment,the respective direction data are subjected to various corrections.Methods of calculating the real space reference direction and thecurrent direction according to the third embodiment will be describedlater.

The operation direction data Dcf includes virtual world referencedirection data Dcf1, operation instruction direction data Dcf2, and thelike. The virtual world reference direction data Dcf1 is datarepresenting a virtual world reference direction set in the virtualworld. The operation instruction direction data Dcf2 is datarepresenting an operation instruction direction which is currentlyinstructed in the virtual world by the user. Methods for calculating thevirtual world reference direction and the operation instructiondirection of the third embodiment will be described later.

The action/attitude data Dcg includes body attitude data Dcg1 and turretattitude data Dcg2, and the like. The body attitude data Dcg is datarepresenting the action and the attitude of the first player object Po1(tank body) in the virtual world. The turret attitude data Dcg2 is datarepresenting the action and the attitude of the second player object Po1(turret) in the virtual world.

The movement vector data Dch is data representing the movement speedsand the movement directions of the respective player objects (the firstplayer object Po1 and the second player object Po2) and the shell objectas in the first embodiment, although the calculation method thereof isdifferent from that of the first embodiment as described later.

The position data Dci is data representing the positions of the playerobject Po, the target object To, the shell object and the like in thevirtual world. The virtual camera data Dcj is data relating to thevirtual camera as in the first embodiment although the calculationmethod thereof is different from that of the first embodiment asdescribed later.

The image data Dck includes player object data Dck1, background imagedata Dck2, and the like. The player object data Dck1 is data forarranging the player object Po (the first player object Po1 and thesecond player object Po2), the target object To, and the shell object ofthe third exemplary game in the virtual world to generate a game image.The background image data Dck2 is data for arranging a background of thethird exemplary game in the virtual world to generate a game image.

Next, the process of the third embodiment, which is performed by thegame apparatus body 5, will be described in detail with reference toFIGS. 28 and 29. FIG. 28 is a sub-routine illustrating an example of aplayer object setting process of the third embodiment executed by thegame apparatus body 5. FIG. 29 is a sub-routine illustrating an exampleof an operation instruction direction calculation process at step 201 inFIG. 28. In the flowcharts of FIGS. 28 and 29, process steps for causingthe player object Po to move and displaying the same, in accordance withan operation performed by a user using the terminal device 6 and theboard type controller 9, will be mainly described among the wholeprocess, while detailed description of other process steps is not given.Further, in FIGS. 28 to 29, each of the steps executed by the CPU 10 isabbreviated as “S”.

When the game apparatus body 5 is powered on, the CPU 10 of the gameapparatus body 5 executes a boot program stored in the ROM/RTC 13 toinitialize each unit such as the main memory. Then, the informationprocessing program stored in the optical disc 4 is loaded to the mainmemory, and the CPU 10 starts to execute the program. The process of thethird embodiment to be performed after completion of the above-mentionedprocess steps is identical to the process of the main flow of the firstembodiment shown in FIG. 12 except the reference direction settingprocess at step 44 and the game control process at step 45. Therefore,the identical part of the process is not repeatedly described. The gamecontrol process at step 44 according to the third embodiment isidentical to the sub-routine of the first embodiment shown in FIG. 13except the player object setting process at step 83, and except that themovement trajectory setting process at step 85 is not performed.Therefore, the identical part of the process is not repeatedlydescribed.

In the main flow of the third embodiment, in the reference directionsetting process at step 41, a reference direction is set after theattitude and the direction of the terminal device 6 are subjected tovarious corrections, and the real space reference direction data Dce1 isupdated by using the reference direction after the corrections. Further,the real space reference direction after the corrections is convertedinto a model coordinate system in the virtual world, and the virtualworld reference direction data Dcf1 is updated by using the directionafter the conversion, as the reference direction in the virtual world.The methods for correcting the attitude and the direction of theterminal device 6 will be described later. Hereinafter, the playerobject setting process at step 83 according to the third embodiment willbe described with reference to FIG. 28.

In FIG. 28, the CPU 10 performs an operation instruction directioncalculation process (step 201), and proceeds to the next step.Hereinafter, the operation instruction direction calculation process atstep 201 will be described with reference to FIG. 29.

In FIG. 29, the CPU 10 corrects the forward direction of the terminaldevice 6 with respect to the upward and downward directions (step 221),and proceeds to the next step. For example, the CPU 10 corrects thedirection (attitude) of the terminal device 6 such that a state in whichthe terminal device 6 tilts downward at a predetermined angle (e.g.,42°) with respect to the horizontal direction in the real space becomesthe operation instruction direction in the horizontal direction.Specifically, in step 82, the CPU 10 calculates X-axis, Y-axis, andZ-axis directions of the terminal device 6 in the real space, based onthe acceleration represented by the acceleration data Dcb1 and theangular velocity represented by the angular velocity data Dcb2, andupdates the current direction data Dce2 by using the calculated X-axis,Y-axis, and Z-axis directions as current directions. Then, in step 221,the CPU 10 corrects the Y-axis direction and the Z-axis direction suchthat the Y-axis direction and the Z-axis direction tilt upward aroundthe X axis by the above-mentioned predetermined angle (i.e., the Y-axisdirection and the Z-axis direction are rotated to the right by thepredetermined angle around the X axis as viewed in the X-axisdirection), by using the X-axis, Y-axis, and Z-axis directionsrepresented by the current direction data Dce2.

Next, the CPU 10 corrects the tilt of the terminal device 6 around the Zaxis (step 222), and proceeds to the next step. For example, the CPU 10corrects the direction (attitude) of the terminal device 6 such that theX axis of the terminal device 6 becomes the horizontal direction in thereal space. Specifically, the CPU 10 forcibly corrects the X-axisdirection to the horizontal direction in the real space by using theX-axis, Y-axis, and Z-axis directions corrected in step 221, and newlycalculates a Z-axis direction, based on a cross product of the X-axisdirection and the Y-axis direction. Then, the CPU 10 newly calculates aY-axis direction, based on a cross product of the newly calculatedZ-axis direction and the X-axis direction corrected to the horizontaldirection.

Next, the CPU 10 performs a correction to limit the tilt of the terminaldevice 6 in the up-down direction of the terminal device 6 (step 223),and proceeds to the next step. For example, the CPU 10 limits the Z-axisdirection newly calculated at step 222 into a predetermined range withrespect to the up-down direction in the real space. If the Z-axisdirection is outside the range, the CPU 10 resets the Z-axis directionin the closest range. Then, the CPU 10 newly calculates a Y-axisdirection, based on a cross product of the reset Z-axis direction andthe X-axis direction corrected to the horizontal direction at step 222.Further, the CPU 10 newly calculates an X-axis direction, based on across product of the newly calculated Y-axis direction and the resetZ-axis direction. Then, the CPU 10 updates the current direction dataDce2 by using the newly calculated X-axis direction and Y-axisdirection, and the reset Z-axis direction.

Also in the reference direction setting process at step 41, the X-axis,Y-axis, and Z-axis directions of the terminal device 6 are corrected asin steps 221 to 223, and the real space reference direction data Dce1 isupdated by using the corrected Z-axis direction as the real spacereference direction.

Next, the CPU 10 calculates a horizontal angular difference between thereal space reference direction and the current direction (step 224), andproceeds to the next step. The horizontal angular difference is anangular difference obtained by projecting an angular difference betweenthe real space reference direction in the real space and the Z-axisdirection represented by the current direction data Dce2 on thehorizontal plane. The horizontal angular difference indicates an angleat which the direction of the terminal device 6 (the direction in whichthe back surface of the terminal device 6 faces (the z-axis positivedirection shown in FIG. 3)) is changed from the initial attitude of theterminal device 6 around the vertical direction in the real space. Forexample, the CPU 10 calculates the horizontal angular difference byusing the real space reference direction represented by the real spacereference direction data Dce1, and the Z-axis direction represented bythe current direction data Dce2.

Next, the CPU 10 calculates an up-down angular difference between thehorizontal direction in the real space and the current direction (step225), and proceeds to the next step. For example, the CPU 10 calculates,as an up-down angular difference, an angular difference between thehorizontal direction in the real space and the Z-axis direction, byusing the Z-axis direction represented by the current direction dataDce2.

Next, the CPU 10 calculates an operation instruction directioncorresponding to the virtual world reference direction, in accordancewith the horizontal angular difference calculated at step 224 and theup-down angular difference calculated at step 225 (step 226), and endsthe process of this sub-routine. For example, based on the virtual worldreference direction represented by the virtual world reference directiondata Dcf1, the CPU 10 calculates an operation instruction direction inthe virtual world such that an angular difference that occurs when thevirtual world reference direction and the operation instructiondirection are projected on the horizontal plane in the virtual world isequal to the horizontal angular difference, and the same positionalrelation is achieved between the real space and the virtual world (i.e.,if the Z-axis direction rotates to the left with respect to the realspace reference direction, the operation instruction direction alsorotates to the left with respect to the virtual world referencedirection). Further, the CPU 10 calculates an operation instructiondirection in the virtual world such that an angular difference betweenthe horizontal direction in the virtual world and the operationinstruction direction is equal to the up-down angular difference, andthe same positional relation is achieved between the real space and thevirtual world (i.e., if the Z-axis direction is downward with respect tothe horizontal direction of the real space, the operation instructiondirection is also downward with respect to the horizontal direction inthe virtual world). Then, the CPU 10 updates the operation instructiondirection data Dcf2 by using the calculated operation instructiondirection.

Referring back to FIG. 28, after the operation instruction directioncalculation process at step 201, the CPU 10 determines whether or notthe operation instruction direction is within a first operation range(step 202). When the operation instruction direction is within the firstoperation range, the CPU 10 proceeds to step 203. On the other hand,when the operation instruction direction is outside the first operationrange, the CPU 10 proceeds to step 204. The first operation range is arotation angle range in the horizontal direction, in which the directionof the second player object Po2 and the position of the target object Toare changed in accordance with the operation instruction direction. Forexample, the first operation range is set to a range from −5° to +5°centering around the virtual world reference direction. Then, in step202, the CPU 10 determines whether or not an angular difference whichoccurs when the virtual world reference direction and the operationinstruction direction are projected on the horizontal plane in thevirtual world is within the first operation range, by using the virtualworld reference direction represented by the virtual world referencedirection data Dcf1 and the operation instruction direction representedby the operation instruction direction data Dcf2. The first operationrange corresponds an example of the turret/target direction operationrange shown in FIG. 26.

In step 203, the CPU 10 sets the direction (turret angle) of the secondplayer object Po2 and the position (target position) of the targetobject To in accordance with the operation instruction direction, andproceeds to step 209. For example, the CPU 10 sets, as the turretdirection of the second player object Po2, the operation instructiondirection represented by the operation instruction direction data Dcf2,and updates the turret attitude data Dcg2 by using the set turretdirection. Based on the turret direction of the second player object Po2and the position of the player object Po represented by the positiondata Dci, the CPU 10 calculates, as a position of the target object To,a position at which a shell object shot from the second player objectPo2 at a predetermined shell speed lands in the virtual world. Then, theCPU 10 updates, by using the calculated position, the position of thetarget object To represented by the position data Dci.

On the other hand, in step 204, the CPU 10 determines whether or not theoperation instruction direction is within a second operation range. Whenthe operation instruction direction is within the second operationrange, the CPU 10 proceeds to step 205. On the other hand, when theoperation instruction direction is outside the second operation range,the CPU 10 proceeds to step 207. The second operation range is arotation angle range in the horizontal direction, in which the directionof the second player object Po2 and the position of the target object Toare changed in accordance with the operation instruction direction, andfurther, the forward direction (body direction) of the first playerobject Po1 is changed so as to approach the operation instructiondirection by a predetermined angle. For example, the second operationrange includes a range from −87° to −5° and a range from +5° to +87°centering around the vertical world reference direction. Then, in step204, the CPU 10 determines whether or not an angular difference whichoccurs when the virtual world reference direction and the operationinstruction direction are projected on the horizontal plane in thevirtual world is within the second operation range, by using the virtualworld reference direction represented by the virtual world referencedirection data Dcf1 and the operation instruction direction representedby the operation instruction direction data Dcf2. The second operationrange corresponds an example of the body direction operation range shownin FIG. 26.

In step 205, the CPU 10 sets the direction (turret angle) of the secondplayer object Po2 and the position (target position) of the targetobject To in accordance with the operation instruction direction, andproceeds to the next step. For example, the CPU 10 sets the operationinstruction direction represented by the operation instruction directiondata Dcf2, as the turret direction of the second player object Po2, andupdates the turret attitude data Dcg2 by using the set turret. Based onthe turret direction of the second player object Po2 and the position ofthe player object Po represented by the position data Dci, the CPU 10calculates, as a position of the target object To, a position at which ashell object shot from the second player object Po2 at a predeterminedshell speed lands in the virtual world. Then, the CPU 10 updates, byusing the calculated position, the position of the target object Torepresented by the position data Dci.

Next, the CPU 10 sets the forward direction (body direction) of thefirst player object Po1 in accordance with the operation instructiondirection (step 206), and proceeds to step 209. For example, based onthe forward direction of the first player object Po1 represented by thebody attitude data Dcg1 and the operation instruction directionrepresented by the operation instruction direction data Dcf2, the CPU 10calculates an angular difference Od which occurs when the forwarddirection of the first player object Po1 and the operation instructiondirection are projected on the horizontal plane in the virtual world.Then, CPU 10 calculates an angle θc2 at which the forward direction ofthe first player object Po1 is changed, in accordance with the followingequation:

θc2=1.0−cos θd

Then, the CPU 10 changes the forward direction of the first playerobject Po1 so as to approach the operation instruction direction by thecalculated angle θc2, and updates the body attitude data Dcg1 by usingthe changed forward direction. Further, since the line-of-sightdirection of the virtual camera for taking a game image to be displayedon the LCD 61 is set to a direction in which the first player object Po1faces, the CPU 10 changes the virtual world reference directionrepresented by the virtual world reference direction data Dcf1, to thesame direction and by the same angle θc2 as the forward direction of thefirst player object Po1, and updates the virtual world referencedirection data Dcf1 by using the changed virtual world referencedirection.

On the other hand, when it is determined at step 204 that the operationinstruction direction is outside the second operation range, the CPU 10determines that the operation instruction direction is within a thirdoperation range. Then, the CPU 10 sets the direction (body direction) ofthe first player object Po1 in accordance with the operation instructiondirection (step 207), and proceeds to the next step. The third operationrange is a rotation angle range in the horizontal direction, in whichthe forward direction (body direction) of the first player object Po1 ischanged at an angle relatively greater than that in the second operationrange. For example, the third operation range includes a range from−180° to −87° and a range from +87° to +180° centering around thevirtual world reference direction. The third operation range correspondsan example of the body direction operation range shown in FIG. 26, and acombination of the second operation range and the third operation rangecorresponds to an example of the body direction operation range shown inFIG. 26.

In step 207, the CPU 10 changes the forward direction of the firstplayer object Po1 so as to approach the operation instruction directionby a predetermined angle θc3 (e.g., 1.0°), based on the forwarddirection of the first player object Po1 represented by the bodyattitude data Dcg1 and the operation instruction direction representedby the operation instruction direction data Dcf2, and updates the bodyattitude data Dcg1 by using the changed forward direction. Since theline-of-sight direction of the virtual camera for taking a game image tobe displayed on the LCD 61 is set to the direction in which the firstplayer object Po1 faces, the CPU 10 changes the virtual world referencedirection represented by the virtual world reference direction dataDcf1, to the same direction and by the same angle θc3 as the forwarddirection of the first player object Po1, and updates the virtual worldreference direction data Dcf1 by using the changed virtual worldreference direction.

Next, the CPU 10 sets the direction (turret angle) of the second playerobject Po2 and the position (target position) of the target object To inaccordance with the body direction (step 208), and proceeds to step 209.For example, in the third operation range, the direction of the secondplayer object Po2 (turret) arranged on the first player object Po1changes with the direction change of the first player object Po1 whilemaintaining the positional relation with the first player object Po1,and further, the position of the target object To also changes with thedirection change of the first player object Po1. Accordingly, the CPU 10changes the turret direction of the second player object Po2 representedby the turret attitude data Dcg2, to the same direction and by the sameangle θc3 as the forward direction of the first player object Po1, andupdates the turret attitude data Dcg2 by using the changed turretdirection. Then, based on the turret direction of the second playerobject Po2 and the position of the player object Po represented by theposition data Dci, the CPU 10 calculates, as a position of the targetobject To, a position at which a shell object shot from the secondplayer object Po2 at a predetermined shell speed lands in the virtualworld. Then, the CPU 10 updates, by using the calculated position, theposition of the target object To represented by the position data Dci.That is, in the process at step 208, the turret direction of the secondplayer object Po2 rotates in the virtual world together with the firstplayer object Po1 while maintaining an angle (e.g., −87° or +87°)corresponding to the boundary between the second operation range and thethird operation range with respect to the first player object Po1.

In step 209, the CPU 10 sets a movement vector based on thecenter-of-gravity position and the body direction, and proceeds to thenext step. For example, the CPU 10 calculates a movement speed of theplayer object Po based on the latest center-of-gravity positionrepresented by the center-of-gravity position data Dcd. For example, theCPU 10 calculates a movement speed of the player object Po, based on aY1 axis value (a coordinate value in a short-side direction) of thecenter-of-gravity position in a predetermined coordinate systemcorresponding to a position on the platform 9 a of the board typecontroller 9. For example, when the center of the platform 9 a is set atthe origin of the coordinate system, if the Y1 axis value of thecenter-of-gravity position is greater than a predetermined numericalvalue (i.e., if the center-of-gravity position is provided at a positionapart from the center of the platform 9 a, forward or backward, by apredetermined distance or more), the CPU 10 sets a forward or backwardmovement speed of the player object Po in accordance with the numericalvalue. Specifically, when the front side of the board type controller 9(the side on which the load sensors 94 c and 94 d are provided as shownin FIG. 6) is set in the Y1 axis direction, if the Y1 axis direction ofthe center-of-gravity position is a positive value and its absolutevalue is greater than the predetermined numerical value, the CPU 10 setsa forward movement speed (e.g., a positive movement speed) of the playerobject Po in accordance with the numerical value. On the other hand, ifthe Y1 axis value of the center-of-gravity position is a negative valueand its absolute value is greater than the predetermined numericalvalue, the CPU 10 sets a backward movement speed (e.g., a negativemovement speed) of the player object Po in accordance with the numericalvalue.

When the forward movement speed (e.g., the positive movement speed) ofthe player object Po is set, the CPU 10 regards the forward direction ofthe first player object Po1 represented by the body attitude data Dcg1,as the movement direction of the player object Po. When the backwardmovement speed (e.g., the negative movement speed) of the player objectPo is set, the CPU 10 regards a direction opposite to the forwarddirection of the first player object Po1 represented by the bodyattitude data Dcg1, as the movement direction of the player object Po.Then, the CPU 10 calculates a movement vector in the virtual world,based on the calculated movement speed and movement direction of theplayer object Po, and updates the movement vector data Dch by using thecalculated movement vector. For example, the CPU 10 sets a length of themovement vector by using the calculated movement speed of the playerobject Po, and sets a direction of the movement vector by using theforward direction of the first player object Po1 represented by the bodyattitude data Dcg1 or a direction opposite to the forward direction.

Next, the CPU 10 calculates a position of the player object Po (step210), and proceeds to the next step. For example, the CPU 10 causes theposition of the player object Po represented by the position data Dci toshift in the virtual world, based on the movement vector represented bythe movement vector data Dch, and updates the position data Dci by usingthe position after the shifting.

Next, the CPU 10 determines whether or not the player object Po isduring shelling (step 211). When the player object Po is duringshelling, the CPU 10 proceeds to step 212. On the other hand, if theplayer object Po is not during shelling, the CPU 10 ends the process ofthis sub-routine. The player object Po is capable of shooting a shellobject from the second player object Po2 toward the target object To,and a shell is shot from the turret toward the set turret direction inaccordance with an operation by the user (e.g., an operation to press anattack button (operation button 64E) of the terminal device 6). The“during shelling” means a period from when the operation to cause theplayer object Po to shoot a shell object toward the target object To isperformed to when the shell object collides with another object or thelike and disappears.

In step 212, the CPU 10 calculates a position and a movement vector ofthe shell object, and ends the process of this sub-routine. For example,when the above-mentioned operation to shoot the shell object has beenperformed, the CPU 10 sets the position of the shell object in theturret as the second player object Po2, and sets a movement vectorhaving a predetermined shell speed in the turret direction, therebyupdating the movement vector of the shell object represented by themovement vector data Dch, and the position of the shell objectrepresented by the position data Dci. After the shell object was shot,the CPU 10 moves the shell object in the virtual world, based on the setmovement vector, to set a new position of the shell object, and updatesthe position of the shell object represented by the position data Dci byusing the set position. Further, the CPU 10 corrects the movementvector, based on the environment (gravity, wind, influence of otherobjects, and the like) set in the virtual world, and updates themovement vector of the shell object represented by the movement vectordata Dch by using the corrected movement vector.

In the virtual camera setting process (step 84) of the third embodiment,the first virtual camera for generating a terminal game image is setsuch that a state of the virtual world viewed from near the back of thefirst player object Po1 arranged in the virtual world is included in theterminal game image. On the other hand, the second virtual camera forgenerating a monitor game image is set in the same virtual world as thefirst virtual camera, such that a state of the virtual world in whichthe player object Po arranged in the virtual world is viewed from adistance, in the forward direction of the first player object Po1, in amanner of bird's eye viewing is included in the monitor game image. TheCPU 10 set the positions of the first virtual camera and the secondvirtual camera in the virtual world (the parameters relating to thefirst virtual camera and the second virtual camera), based on theposition of the player object Po represented by the position data Dbiand the forward direction of the first player object Po1 represented bythe body attitude data Dcg1.

In the monitor game image generation process (step 45) and the terminalgame image generation process (step 46) of the third embodiment, imagesrepresenting the result of the game control process are generated,respectively, as in the first embodiment. For example, the monitor gameimage is generated as a three-dimensional CG image which is generatedthrough the steps of: arranging the second virtual camera in the virtualworld based on the parameters relating to the second virtual camera,which are represented by the virtual camera data Dcj; arranging theplayer object Po (the first player object Po1 and the second playerobject Po2), the target object To, the shell object, and the like in thevirtual world based on the action/attitude data Dcg and the positiondata Dci; and calculating a scene of the virtual world viewed from thesecond virtual camera. The terminal game image is generated as athree-dimensional CG image which is generated through the steps of:arranging the first virtual camera in the virtual world based on theparameters relating to the first virtual camera, which are representedby the virtual camera data Dcj; arranging the player object Po, thetarget object To, the shell object, and the like in the virtual worldbased on the action/attitude data Dcg and the position data Dci; andcalculating a scene of the virtual world viewed from the first virtualcamera.

In this way, according to the above-mentioned process, in the operationusing the terminal device 6 within the turret/target direction operationrange (the first operation range), only the direction of the secondplayer object Po2 and the position of the target object To change, whilethe direction of the first player object Po1 and the display range ofthe virtual world displayed on the LCD 61 do not change. Accordingly, inthe operation using the terminal device 6 within the turret/targetdirection operation range (the first operation range), the objects to bemoved by this operation are limited, and favorable operation environmentis provided for adjusting the position of the target object To to shoota shell toward the enemy object Eo or the like. Further, in theoperation using the terminal device 6 within the body directionoperation range (the second operation range or the third operationrange) in the real space, not only the direction of the second playerobject Po2 and the position of the target object To but also thedirection of the first player object Po1 and the position and thedirection of the virtual camera can be changed. Accordingly, theoperation using the terminal device 6 within the body directionoperation range enables an operation of the objects other than theobjects operable in the turret/target direction operation range, and isfavorable for changing the movement direction of the player object Po,or changing the display range on the LCD 61. Thus, setting at least twooperation ranges allows the user to perform various operations by usingthe attitude of a single device.

In the above-mentioned first to third exemplary games, control (controlof the position and the direction) of the virtual camera (the firstvirtual camera) for generating an image to be displayed on the LCD 61 isperformed based on the attitude of the terminal device 6. Such controlprovides the user with an image that makes the user feel like lookinginto the virtual world through the LCD 61, and thereby allows the userto experience a sense of presence in the virtual world. Further, theoperation based on the attitude of the terminal device 6 enablesrotation operations in three directions, i.e., the left-right swingmotion (yaw) around the vertical direction (e.g., around the Y-axisdirection), the up-down swing motion (pitch) around the left-righthorizontal direction (e.g., around the X-axis direction), and theleft-right rotation (roll) around the front-rear horizontal direction(e.g., around the Z-axis direction). Therefore, the operation based onthe attitude of the terminal device 6 is suitable for control of thevirtual camera that is similarly movable in the virtual world. Bycontrolling the attitude of the virtual camera in the virtual world soas to be equal to the attitude of the terminal device 6 in the realspace, it is possible to provide the user with an image that makes theuser feel like looking into the virtual world along his/her desireddirection. Further, in the first to third exemplary games, the playerobject performs an action (e.g., moves) in accordance with an action ofthe user on the board type controller 9. That is, the image displayed onthe LCD 61 allows the user to experience a sense of presence in thevirtual world, and moreover, allows the user to experience an operationsense of becoming the player character in the real space, which enhancesthe sense of presence in the virtual world.

In the first to third exemplary games, the attitude of the player objectPo displayed on the LCD 61 is controlled based on the attitude of theterminal device 6. Such control provides the user with an operationenvironment as if the terminal device 6 is the player object Po, andallows the user to experience a sense of becoming the player object Poin the virtual world, and a sense of directly operating the playerobject Po. Further, the operation based on the attitude of the terminaldevice 6 enables rotation operations in three directions, i.e., theleft-right swing motion (yaw) around the vertical direction (e.g.,around the Y-axis direction), the up-down swing motion (pitch) aroundthe left-right horizontal direction (e.g., around the X-axis direction),and the left-right rotation (roll) around the front-rear horizontaldirection (e.g., around the Z-axis direction). Therefore, the operationbased on the attitude of the terminal device 6 is suitable for controlof the player object Po that is similarly movable in the virtual world.For example, in the first exemplary game, the left-right swing motion(yaw) around the vertical direction (Y-axis direction) along the LCD 61of the terminal device 6 is related to an attitude change (yaw) of theplayer object Po in the left-right direction, the up-down swing motion(pitch) around the left-right direction (X-axis direction) along the LCD61 is related to an attitude change (pitch) of the player object Po inthe up-down direction, and the left-right rotation (roll) around thedepth direction (Z-axis direction) of the LCD 61 is related to arotational attitude change (roll) around the forward direction of theplayer object Po, thereby providing the user with an image of an objectwhose attitude changes as desired by the user in the virtual world.Further, in the first to third exemplary games, the player objectperforms an action (e.g., moves) in accordance with an action of theuser on the board type controller 9. That is, the user operates oneplayer object by using a plurality of devices (the terminal device 6 andthe board type controller 9), which realizes a new operation. Further,control of the attitude of the player object based on the attitude ofthe terminal device 6 allows the user to experience a sense of becomingthe player object in the virtual world, and a sense of directlyoperating the player object. Moreover, user's operation on the boardtype controller 9 allows the user to experience a sense of becoming theplayer character in the real space, which enhances the sense of presencein the virtual world. Further, in the first to third exemplary games, itis possible to move the player object in the direction forward from theplayer object. Accordingly, the user is allowed to set, based on theattitude of the terminal device 6, the movement direction of the playerobject as well as the attitude (forward direction) of the player object,and thus the user is allowed to easily and intuitively set the movementdirection of the player object to his/her desired direction. Further, aswell as the attitude of the player object, it is possible to set themovement direction of the player object in accordance with the rotationoperations in the three directions of the terminal device 6. Therefore,the user is allowed to set his/her desired movement direction as it isin the virtual world.

In the first to third exemplary games, the movement direction of theplayer object Po is set based on the attitude of the terminal device 6.Such setting of the movement direction provides the user with anoperation environment as if the terminal device 6 becomes the playerobject Po, and allows the user to experience a sense of presence in thevirtual world as the player object Po, and a sense of directly operatingthe player object Po. Further, the operation using the attitude of theterminal device 6 enables rotation operations in three directions, i.e.,the left-right swing motion (yaw) around the vertical direction (e.g.,around the Y-axis direction), the up-down swing motion (pitch) aroundthe left-right horizontal direction (e.g., around the X-axis direction),and the left-right rotation (roll) around the front-rear horizontaldirection (e.g., around the Z-axis direction). Therefore, the operationbased on the attitude of the terminal device 6 is suitable for controlof the player object Po whose movement direction can be set in at leastone direction among the three directions in the virtual world. Forexample, in the case of the first exemplary game, the left-right swingmotion (yaw) around the vertical direction (Y-axis direction) along theLCD 61 of the terminal device 6 is related to a movement directionchange (yaw) of the player object Po in the left-right direction, andthe up-down swing motion (pitch) around the left-right direction (X-axisdirection) along the LCD 61 is related to a movement direction change(pitch) of the player object Po in the up-down direction, therebyproviding an object control for causing the player object Po to move ina movement direction desired by the user in the virtual world. Further,in the first to third exemplary games, setting of the movement (e.g.,setting of the movement speed) of the player object is performed inaccordance with an action of the user on the board type controller 9.That is, the user controls the movement direction and the movement speedof one player object by using a plurality of devices (the terminaldevice 6 and the board type controller 9), thereby realizing a newoperation. Further, setting of the movement direction of the playerobject based on the attitude of the terminal device 6 allows the user toexperience a sense of becoming the player object in the virtual world,and a sense of directly operating the player object. Moreover, theuser's operation on the board type controller 9 allows the user toexperience an operation sense of becoming the player object in the realspace, which enhances the sense of presence in the virtual world.

In the first to third exemplary games, the direction of the virtualworld displayed on the LCD 61 of the terminal device 6 is the movementdirection of the player object. Accordingly, the user is allowed to set,based on the attitude of the terminal device 6, the movement directionof the player object. Further, since the virtual world viewed in themovement direction is displayed on the LCD 61, the user is allowed toeasily and intuitively set the movement direction of the player objectto his/her desired direction. Further, as well as the direction of thevirtual camera, it is possible to set the movement direction of theplayer object in accordance with the rotation operations in the threedirections of the terminal device 6, the user is allowed to set his/herdesired movement direction as it is in the virtual world.

Further, in the first to third exemplary games, an image of a virtualworld including at least the player object Po is displayed on the LCD 61of the terminal device 6. However, an image of another type of a virtualworld may be displayed on the LCD 61. For example, an image of a virtualworld viewed from a subjective point of the player object Po may bedisplayed on the LCD 61 without displaying the player object Po. In thiscase, the direction of the virtual world displayed on the LCD 61 fromthe subjective point may be the same as the movement direction of theplayer object Po, or may be different from the movement direction of theplayer object Po. When the direction of the virtual world displayed onthe LCD 61 from the subjective point is the same as the movementdirection of the player object Po, the virtual world viewed in themovement direction is displayed on the LCD 61. Therefore, the user isallowed to easily and intuitively set the movement direction of theplayer object to his/her desired direction.

The above-mentioned first to third exemplary games adopts an example ofa process in which control of the position and the attitude of thevirtual camera is immediately associated with the operation instructiondirection determined based on the attitude of the terminal device 6.However, control of the position and the attitude of the virtual cameramay be delayed for a predetermined time in accordance with a change inthe operation instruction direction. In this case, after the directionof the player object has changed, the position and the direction of thevirtual camera follows the direction of the player object after thepredetermined time.

Further, in the first to third exemplary games, the game image displayedon the LCD 61 of the terminal device 6 and the game image displayed onthe monitor 2 are images showing the same virtual world (virtual space),but the images are viewed from different viewpoints or in differentranges of the virtual world. Accordingly, the user is allowed to viewdifferent views or ranges of the virtual world (virtual space) displayedon the two display screens, and thereby allowed to view appropriate gameimages according to the game situation or the like. Further, in thefirst to third exemplary games, the user holds and operates the terminaldevice 6 to change the position and the attitude of the player object orthe virtual camera in accordance with the attitude and the position ofthe terminal device 6 in the real space, and change the image displayedon the LCD 61 in accordance with the position and the attitude of theplayer object or the virtual camera. Accordingly, the user, who viewsthe image displayed on the LCD 61 with the terminal device 6 in his/herhands, is given a sense of presence in the virtual world (virtualspace). On the other hand, it is difficult for the user, who views onlythe image displayed on the LCD 61, to know the position of the playerobject in the entire virtual world (virtual space) and the surroundingsof the player object. Such a problem is resolved by displaying, on themonitor 2, a relatively wide range of the virtual world (virtual space).

In the first to third exemplary games, the user is allowed to perform anoperation by changing a load applied to the board type controller 9 bythe user, and further allowed to perform an operation (an operationaccording to the attitude or the position of the terminal device 6, atouch operation, a button operation, or the like) with holding theterminal device 6 on which a game image including a player object isdisplayed. The player object displayed on the terminal device 6 iscaused to move in the virtual world (virtual space) in accordance withthe operation of changing the load applied to the board type controller9 and/or the operation using the terminal device 6. Accordingly, theuser is allowed to experience a sense of acting as the player object andviewing the virtual world (virtual space), and a sense of operating theplayer object in the real world.

In the operation using the board type controller 9, presence/absence ofa load applied to the board type controller 9, a change in a load, and acenter-of-gravity position of a load are used for various operationcontrols for the player objects. For example, in the first to thirdexemplary games, the action, the movement speed, the movement speed andthe like of the player object are changed in accordance with a change inthe load applied to the board type controller 9, and thecenter-of-gravity position of the load. In the exemplary games, althoughthe direction (e.g., the movement direction or the forward direction ofthe player object) in the virtual world (virtual space) is set inaccordance with the direction of the user's operation, the direction isset based on the attitude (direction) of the terminal device 6. Further,correction of the movement direction in the air in the virtual world inthe second exemplary game, and switching between the forward movementand the backward movement in the third exemplary game, are performedbased on the center-of-gravity position of the load applied to the boardtype controller 9. That is, between the operation based on the attitudeof the terminal device 6 and the operation using the board typecontroller 9, an appropriate operation, which facilitates setting of adirection or enhances a sense of presence in the virtual world, isselected. Thus, an appropriate one, between the attitude of the terminaldevice 6 and the center-of-gravity position of the load applied to theboard type controller 9, can be selected according to the game content,as an operation to instruct a direction. In this way, in the case wherethe terminal device 6 and the board type controller 9 are used asoperation means, when setting a direction in the virtual world (virtualspace) in accordance with an operation of the user, the user is allowedto select an appropriate operation among a plurality of operations asoptions.

The game system 1 allows the user to play various games by using theterminal device 6 and the board type controller 9 as operation means.The terminal device 6 serves as a controller which allows the user toperform an input by an operation based on its motion, a touch operation,a button operation or the like, while it also serves as a portabledisplay or a second display. Therefore, the game system 1 realizes awide range of games. That is, since the terminal device 6 functions as adisplay device, there may be a game system in which the terminal device6 is used as display means and the board type controller 9 is used asoperation means while the monitor 2 and the controller 7 are not used.Further, since the terminal device 6 functions as an operation device aswell as a display device, there may be a game system in which theterminal device 6 is used as display means and the terminal device 6 andthe board type controller 9 are used as operation means while themonitor 2 and the controller 7 are not used. Furthermore, since theterminal device 6 functions as an operation device as well as a displaydevice, there may be a game system in which the terminal device 6 isused as display means and operation means while the monitor 2, thecontroller 7, and the board type controller 9 are not used.

In the above-mentioned embodiments, the terminal device 6 functions as aso-called thin client terminal which does not execute a game process.However, at least a part of the sequential game processes executed bythe game apparatus body 5 may be executed by the terminal device 6. Forexample, the terminal game image generating process may be executed bythe terminal device 6. In another example, all the sequential gameprocesses to be executed by the game apparatus body 5 may be executed bythe terminal device 6. In this case, since the terminal device 6functions as a processing device for executing the game processes aswell as a display device, there may be a game system in which theterminal device 6 is used as display means, the board type controller 9is used as operation means, and the terminal device 6 is used asprocessing means while the monitor 2, the game apparatus body 5, and thecontroller 7 are not used. In this game system, only the terminal device6 and the board type controller 9 are connected wirelessly or via acable, and board operation data is transmitted from the board typecontroller 9 to the terminal device 6, thereby realizing various kindsof games. Further, when the board type controller 9 is not used, theremay be a game system in which the terminal device 6 is used as displaymeans, operation means, and processing means.

In the above-mentioned embodiments, the attitude data (e.g., dataoutputted from at least one of the magnetic sensor 602, the accelerationsensor 603, and the gyro sensor 604) used for calculating the movement(including the position and the attitude, or a change in the position orthe attitude) of the terminal device 6 is transmitted from the terminaldevice 6 to the game apparatus body 5, and the movement of the terminaldevice 6 is calculated by the information processing in the gameapparatus body 5. However, the movement of the terminal device 6 to becalculated in the game apparatus body 5 may be calculated in theterminal device 6. In this case, data representing the movement of theterminal device 6, which is calculated in the terminal device 6 (i.e.,data representing the position and the attitude of the terminationdevice, or a change in the position or the attitude, which is calculatedbased on the attitude data) is transmitted from the terminal device 6 tothe game apparatus body 5, and used in the information processing in thegame apparatus body 5.

In the above description, the terminal device 6 and the game apparatusbody 5 are connected by wireless communication, and the board typecontroller 9 and the game apparatus body 5 are connected by wirelesscommunication. However, the inter-device wireless communication may beachieved in a manner other than mentioned above. In a first example, theterminal device 6 serves as a relay device for another wirelesscommunication. In this case, the board operation data of the board typecontroller 9 is wirelessly transmitted to the terminal device 6, and theterminal device 6 wirelessly transmits, to the game apparatus body 5,the terminal operation data of the terminal device 6 together with thereceived board operation data. In this case, while the terminal device 6and the game apparatus body 5 are directly connected by wirelesscommunication, the board type controller 9 is connected via the terminaldevice 6 to the game apparatus body 5 by wireless communication. In asecond example, the board type controller 9 serves as a relay device foranother wireless communication. In this case, the terminal operationdata of the terminal device 6 is wirelessly transmitted to the boardtype controller 9, and the board type controller 9 wirelessly transmits,to the game apparatus body 5, the board operation data of the board typecontroller 9 together with the received terminal operation data. In thiscase, the board type controller 9 and the game apparatus body 5 aredirectly connected by wireless communication, and the terminal device 6is connected via the board type controller 9 to the game apparatus body5 by wireless communication. When another device serves as a relaydevice to transmit operation data to the game apparatus body 5, a devicewhich generates the operation data and the relay device which relays thecontrol data may be electrically connected via a cable.

Further, the terminal device 6 and/or the board type controller 9 may beelectrically connected to the game apparatus body 5 via cables. In thiscase, the cables connected to the terminal device 6 and/or the boardtype controller 9 are connected to a connection terminal of the gameapparatus body 5. In a first example, the terminal device 6 and the gameapparatus body 5 are electrically connected via a first cable, and theboard type controller 9 and the game apparatus body 5 are electricallyconnected via a second cable. In a second example, the terminal device 6is electrically connected to the game apparatus body 5 via a cable. Inthis case, the board operation data of the board type controller 9 maybe wirelessly transmitted to the terminal device 6 and then transmittedto the game apparatus body 5 via the cable. Alternatively, the boardoperation data of the board type controller 9 may be wirelesslytransmitted from the board type controller 9 directly to the gameapparatus body 5. In a third example, the board type controller 9 iselectrically connected to the game apparatus body 5 via a cable. In thiscase, the terminal operation data of the terminal device 6 may bewirelessly transmitted to the board type controller 9 and thentransmitted to the game apparatus body 5 via the cable. Alternatively,the terminal operation data of the terminal device 6 may be wirelesslytransmitted from the terminal device 6 directly to the game apparatusbody 5.

Further, in the above-mentioned embodiments, the game system 1 includesone terminal device 6 and one board type controller 9. However, the gamesystem 1 may be configured to include a plurality of terminal devices 6and a plurality of board type controllers 9. In this case, the gameapparatus body 5 may be wirelessly communicable with the respectiveterminal devices 6 and the respective type controllers 9, and maytransmit game image data, game sound data, and control data to therespective terminal devices, and receive terminal operation data, cameraimage data, microphone sound data, and board operation data from therespective terminal devices 6 and the respective board type controllers9. When the game apparatus body 5 wirelessly communicates with theplurality of terminal devices 6 and the plurality of board typecontrollers 9, the game apparatus body 5 may perform the wirelesscommunications in a time division manner or a frequency division manner.

When the game system 1 includes a plurality of terminal devices 6 and aplurality of board type controllers 9 as described above, a plurality ofusers are allowed to play more types of games. For example, when thegame system 1 includes two pairs of terminal devices 6 and board typecontrollers 9, two users are allowed to play a game simultaneously.Further, when the game system 1 includes two pairs of terminal devices 6and board type controllers 9, since the game system 1 includes threedisplay units, game images for three users can be generated to bedisplayed on the respective display units.

In the above description, the board type controller 9 is provided with aplurality of load sensors 94. However, if information of thecenter-of-gravity position of a load applied to the board typecontroller 9 is not required in the above-mentioned processing, theboard type controller 9 may be provided with at least one load sensor94.

The exemplary embodiments have been described by using the stationarygame apparatus 3. However, the information processing program may beexecuted by an information processing apparatus such as a hand-held gameapparatus or a general personal computer. Further, the exemplaryembodiments may be applied to, as well as the game apparatus, anyhand-held electronic devices, e.g., a PDA (Personal Digital Assistant),a mobile telephone, a personal computer, a camera, and the like. Anydevice may be connected to the terminal device 6 and the board typecontroller 9 wirelessly or via cables.

In the above description, the information processing is executed by thegame apparatus body 5. However, at least a part of the process steps inthe information processing may be performed by another apparatusprovided outside the game system 1. For example, when the game apparatusbody 5 is configured to be communicable with another apparatus (e.g., aserver or another game apparatus), the process steps in the informationprocessing may be performed by the game apparatus body 5 in combinationwith the other apparatus. For example, the other apparatus performs aprocess of setting a player object, a virtual world and the like, anddata relating to an action or an attitude of the player object istransmitted from the game apparatus body 5 to the other apparatus, andthereby the information processing is performed. Then, image datarepresenting a virtual world generated in the other apparatus istransmitted to the game apparatus body 5, and the virtual world isdisplayed on the monitor 2 and the LCD 61. Thus, when at least a part ofthe process steps in the information processing is performed by anotherapparatus, the same processing as the information processing isachieved. At least a part of the process steps in the informationprocessing may be performed by the board type controller 9(microcomputer 100). Further, the above-mentioned information processingcan be executed by one processor or by a cooperation of a plurality ofprocessors which is/are included in an information processing systemconstituted by at least one information processing apparatus. Further,in the exemplary embodiment, the processes shown in the above-mentionedflowcharts are performed by the CPU 10 of the game apparatus body 5executing the predetermined program. However, a part or the entirety ofthe processes may be performed by a dedicated circuit included in thegame apparatus body 5.

Further, the shape of the game apparatus body 5, the shapes of theterminal device 6, the controller 7, and the board type controller 9,and the shapes, numbers, and mounting positions of the operation buttonsand the sensors, are merely examples. It should be understood that othershapes, numbers, and mounting positions may be adopted. Further, theprocessing sequences, set values, display modes, and values used indetermination which are used in the above-mentioned game process aremerely examples. It is understood that other sequences, display modes,and values may be adopted.

Further, the information processing program (game program) may besupplied to the game apparatus body 5 via a wired or wirelesscommunication circuit, as well as to the game apparatus body 5 via anexternal storage medium such as the optical disc 4. Further, theinformation processing program may be previously stored in a nonvolatilestorage device inside the game apparatus body 5. Examples of aninformation storage medium storing the information processing programtherein include: a CD-ROM, a DVD, any other optical disc-shaped storagemedium similar to these media, a flexible disk, a hard disk, a magneticoptical disk, and a magnetic tape. Further, the information storagemedium storing the information processing program therein may be anonvolatile semiconductor memory or a volatile memory. Such a storagemedium is a storage medium readable by a computer or the like. Forexample, it is possible to provide the above-mentioned various functionsby causing a computer or the like to read and execute the programsstored in these storage media.

The systems, devices and apparatuses described herein may include one ormore processors, which may be located in one place or distributed in avariety of places communicating via one or more networks. Suchprocessor(s) can, for example, use conventional 3D graphicstransformations, virtual camera and other techniques to provideappropriate images for display. By way of example and withoutlimitation, the processors can be any of: a processor that is part of oris a separate component co-located with the stationary display and whichcommunicates remotely (e.g., wirelessly) with the movable display; or aprocessor that is part of or is a separate component co-located with themovable display and communicates remotely (e.g., wirelessly) with thestationary display or associated equipment; or a distributed processingarrangement some of which is contained within the movable displayhousing and some of which is co-located with the stationary display, thedistributed portions communicating together via a connection such as awireless or wired network; or a processor(s) located remotely (e.g., inthe cloud) from both the stationary and movable displays andcommunicating with each of them via one or more network connections; orany combination or variation of the above.

The processors can be implemented using one or more general-purposeprocessors, one or more specialized graphics processors, or combinationsof these. These may be supplemented by specifically-designed ASICs(application specific integrated circuits) and/or logic circuitry. Inthe case of a distributed processor architecture or arrangement,appropriate data exchange and transmission protocols are used to providelow latency and maintain interactivity, as will be understood by thoseskilled in the art.

Similarly, program instructions, data and other information forimplementing the systems and methods described herein may be stored inone or more on-board and/or removable memory devices. Multiple memorydevices may be part of the same device or different devices, which areco-located or remotely located with respect to each other.

While the exemplary embodiments have been described in detail, theforegoing description is in all aspects illustrative and notrestrictive. It is understood that numerous other modifications andvariations can be devised. It is also understood that, from thedescription of specific embodiments, the one skilled in the art canimplement the exemplary embodiments in the equivalent range based on thedescription and on the common technological knowledge. Further, itshould be understood that terms used in the present specification havemeanings generally used in the art concerned unless otherwise specified.Therefore, unless otherwise defined, all the jargon and technical termshave the same meanings as those generally understood by one skilled inthe art of the exemplary embodiments. In the event of any conflict, thepresent specification (including meanings defined herein) has priority.

1. A computer-readable storage medium having stored therein aninformation processing program executed on a computer of an informationprocessing apparatus which is capable of displaying an image on aportable display device which outputs at least attitude data based on anattitude of a body of the portable display device, wherein theinformation processing program causes the computer to act as: adirection determination section configured to determine, based on theattitude data, whether or not a direction of a predetermined axis set inthe portable display device is within a first range; a first objectaction control section configured to control an action of a first objectarranged in a virtual world, in accordance with the direction of thepredetermined axis, when the result of the determination is that thedirection of the predetermined axis is within the first range; a secondobject action control section configured to control at least an actionof a second object arranged in the virtual world, in accordance with thedirection of the predetermined axis, when the result of thedetermination is that the direction of the predetermined axis is outsidethe first range; and a display control section configured to display, onthe portable display device, a first image showing the virtual worldincluding at least the first object and the second object.
 2. Thecomputer-readable storage medium according to claim 1, wherein theinformation processing program further causes the computer to act as anattitude calculation section configured to calculate an attitude of theportable display device, based on the attitude data, and the directiondetermination section determines whether or not the direction of thepredetermined axis is within the first range, based on the attitude ofthe portable display device.
 3. The computer-readable storage mediumaccording to claim 2, wherein the first object action control sectioncontrols the action of the first object, based on a change in thedirection of the predetermined axis, and the second object actioncontrol section controls at least the action of the second object, basedon the change in the direction of the predetermined axis.
 4. Thecomputer-readable storage medium according to claim 3, wherein the firstobject action control section controls the action of the first objectsuch that the greater the change in the direction of the predeterminedaxis is, the greater the movement of the first object is, and the secondobject action control section controls the action of the second objectsuch that the greater the change in the direction of the predeterminedaxis is, the greater the movement of the second object is.
 5. Thecomputer-readable storage medium according to claim 2, wherein the firstobject action control section controls the action of the first object bycausing the first object to rotate in accordance with an angle of thepredetermined axis, and the second object action control sectioncontrols the action of the second object by causing the second object torotate in accordance with the angle of the predetermined axis.
 6. Thecomputer-readable storage medium according to claim 2, wherein thesecond object action control section controls the action of the secondobject in accordance with the direction of the predetermined axis, andcontrols the action of the first object in accordance with the action ofthe second object.
 7. The computer-readable storage medium according toclaim 6, wherein the second object action control section controls theaction of the second object in accordance with the direction of thepredetermined axis, and controls the action of the first object suchthat the first object performs the same action as the second object. 8.The computer-readable storage medium according to claim 2, wherein thedirection determination section sets, as the first range, a range withina first angle, and sets, as a second range, a range which is within asecond angle larger than the first angle and is outside the first range,and determines whether or not the direction of the predetermined axis iswithin the second range, and the second object action control sectioncontrols the action of the second object by causing the second object torotate at a first angular velocity in accordance with the direction ofthe predetermined axis, when the result of the determination is that thedirection of the predetermined axis is within the second range, andcontrols the action of the second object by causing the second object torotate at a second angular velocity which is lower than the firstangular velocity, in accordance with the direction of the predeterminedaxis, when the result of the determination is that the direction of thepredetermined axis is neither in the first range nor in the secondrange.
 9. The computer-readable storage medium according to claim 8,wherein the direction determination section sets a reference directionwith respect to the direction of the predetermined axis, and sets, asthe first range, a range within the first angle centering around thereference direction, and sets, as the second range, a range which iswithin the second angle centering around the reference direction and isoutside the first range.
 10. The computer-readable storage mediumaccording to claim 2, wherein the direction determination section sets areference direction with respect to the direction of the predeterminedaxis, and sets, as the first range, a range within the first anglecentering around the reference direction.
 11. The computer-readablestorage medium according to claim 10, wherein the directiondetermination section sets the reference direction on a predeterminedhorizontal plane, and sets, as the first range, a range within the firstangle on the horizontal plane centering around the reference direction.12. The computer-readable storage medium according to claim 2, whereinthe display control section displays, separately from the first image, asecond image showing the virtual world on another display deviceconnected to the information processing apparatus.
 13. Thecomputer-readable storage medium according to claim 12, wherein thedisplay control section displays, on the other display device, as thesecond image, an image of the virtual world including the first objectand the second object, in which the virtual world is viewed from aviewpoint different from the viewpoint of the virtual world of the firstimage.
 14. The computer-readable storage medium according to claim 13,wherein the display control section sets a viewpoint of the virtualworld of the second image, in a position apart from the first object andthe second object by a distance longer than the distance from theviewpoint of the virtual world of the first image to the first objectand the second object, and the display control section displays, on theother display device, as the second image, a range of the virtual world,which is larger than the range of the virtual world shown by the firstimage.
 15. The computer-readable storage medium according to claim 13,wherein the display control section sets a viewpoint for generating thesecond image, in a position in the virtual world from which the firstobject and the second object are viewed in a manner of bird's eyeviewing, and the display control section displays, on the other displaydevice, as the second image, an image in which the first object and thesecond object arranged in the virtual world are viewed in a manner ofbird's eye viewing.
 16. The computer-readable storage medium accordingto claim 12, wherein the display control section sets a first virtualcamera for generating the first image and a second virtual camera forgenerating the second image, and causes a line-of-sight direction of thesecond virtual camera in the virtual world to change in accordance witha change in a line-of-sight direction of the first virtual camera in thevirtual world.
 17. The computer-readable storage medium according toclaim 16, wherein the display control section arranges the first virtualcamera such that a forward direction of the second object from behindthe second object in the virtual world is an imaging direction of thefirst virtual camera, arranges the second virtual camera such that aposition in the forward direction of the second object in the virtualworld is a point of view of the second virtual camera, and the secondobject is displayed in the second image at a constant position withrespect to the point of view, and controls the positions and/or theattitudes of the first virtual camera and the second virtual camera inaccordance with the action of the second object.
 18. Thecomputer-readable storage medium according to claim 2, wherein theattitude calculation section calculates an attitude of the portabledisplay device with respect to the a predetermined direction in the realspace, the direction determination section determines whether or not thedirection of the predetermined axis is within the first range which isset with respect to the predetermined direction in the real space, basedon the attitude of the portable display device with respect to thepredetermined direction in the real space, when the result of thedetermination is that the direction of the predetermined axis withrespect to the predetermined direction in the real space is within thefirst range, the first object action control section controls the actionof the first object, with reference to a predetermined direction whichis set in the virtual world and corresponds to the predetermineddirection in the real space, based on the direction of the predeterminedaxis with respect to the predetermined direction in the real space, andwhen the result of the determination is that the direction of thepredetermined axis with respect to the predetermined direction in thereal space is outside the first range, the second object action controlsection controls the action of the second object with reference to thepredetermined direction set in the virtual world, based on the directionof the predetermined axis with respect to the predetermined direction inthe real space.
 19. The computer-readable storage medium according toclaim 18, wherein the attitude calculation section calculates, with adirection of gravity in the real space being the predetermineddirection, an attitude of the portable display device with respect tothe direction of gravity, the direction determination section sets, withthe direction of gravity in the real space being the predetermineddirection in the real space, the first range with respect to thedirection of gravity, and determines whether or not the direction of thepredetermined axis is within the first range, based on the attitude ofthe portable display device with respect to the direction of gravity inthe real space, when the result of the determination is that thedirection of the predetermined axis is within the first range, the firstobject action control section controls the action of the first objectwith reference to the direction of gravity set in the virtual world,based on the direction of the predetermined axis with respect to thedirection of gravity in the real space, and when the result of thedetermination is that the direction of the predetermined axis is outsidethe first range, the second object action control section controls theaction of the second object with reference to the direction of gravityset in the virtual world, based on the direction of the predeterminedaxis with respect to the direction of gravity in the real space.
 20. Thecomputer-readable storage medium according to claim 19, wherein theattitude calculation section calculates at least an attitude in whichthe portable display device rotates around the direction of gravity inthe real space, the direction determination section sets the first rangein a horizontal direction with respect to the direction of gravity inthe real space, and determines whether or not the direction of thepredetermined axis which is projected onto a horizontal plane in thereal space is within the first range, when the result of thedetermination is that the direction of the predetermined axis projectedonto the horizontal plane in the real space is within the first range,the first object action control section controls the action of the firstobject by causing the first object to rotate and/or move along thehorizontal direction set in the virtual world, based on the direction ofthe predetermined axis projected onto the horizontal plane, and when theresult of the determination is that the direction of the predeterminedaxis projected onto the horizontal plane in the real space is outsidethe first range, the second object action control section controls theaction of the second object by causing the second object to rotateand/or move along the horizontal direction set in the virtual world,based on the direction of the predetermined axis projected onto thehorizontal direction.
 21. The computer-readable storage medium accordingto claim 20, wherein the attitude calculation section further calculatesan attitude in which the portable display device swings up and downaround a horizontal direction perpendicular to the direction of gravityin the real space, and the first object action control section controlsthe action of the first object by causing the first object to rotateand/or move around a horizontal direction which is set in the virtualworld and corresponds to the horizontal direction in the real space,based on the attitude in which the portable display device swings up anddown around the horizontal direction in the real space, regardless ofwhether the result of the determination by the direction determinationsection is positive or negative.
 22. The computer-readable storagemedium according to claim 2, wherein the attitude calculation sectioncalculates at least an attitude in which the portable display devicerotates around two axes orthogonal to the predetermined axis, thepredetermined axis being a depth direction of a display screen of theportable display device, the depth direction being perpendicular to thedisplay screen on which the first image is displayed, the directiondetermination section sets the first range around one of the two axes,and determines whether or not the depth direction rotated around the oneaxis is within the first range, when the result of the determination isthat the depth direction is within the first range, the first objectaction control section causes the first object to rotate and/or movearound an axis of a vertical direction in the virtual world, based onthe depth direction rotated around the one axis, when the result of thedetermination is that the depth direction is outside the first range,the second object action control section causes the second object torotate or move around the axis of the vertical direction in the virtualworld, based on the depth direction which has rotated around the oneaxis, and the first object action control section causes the firstobject to rotate and/or move around the horizontal direction in thevirtual world, based on the depth direction rotated around the otheraxis, regardless of whether the result of the determination by thedirection determination section is positive or negative.
 23. Thecomputer-readable storage medium according to claim 22, wherein theattitude calculation section calculates at least an attitude in whichthe portable display device rotates around an axis along a horizontaldirection of a display screen of the portable display device and an axisalong a vertical direction of the display screen, the axes beingorthogonal to the depth direction, the direction determination sectionsets the first range around the axis along the vertical direction of thedisplay screen, and determines whether or not the depth directionrotated around the axis along the vertical direction is within the firstrange, when the result of the determination is that the depth directionis within the first range, the first object action control sectioncauses the first object to rotate and/or move around the axis of thevertical direction in the virtual world, based on the depth directionrotated around the axis of the vertical direction of the display screen,when the result of the determination is that the depth direction isoutside the first range, the second object action control section causesthe second object to rotate and/or move around the axis of the verticaldirection in the virtual world, based on the depth direction rotatedaround the axis of the vertical direction of the display screen, and thefirst object action control section causes the first object to rotateand/or move around the horizontal direction in the virtual world, basedon the depth direction rotated around the axis of the horizontaldirection of the display screen, regardless of whether the result of thedetermination by the direction determination section is positive ornegative.
 24. The computer-readable storage medium according to claim 1,wherein the information processing program further causes the computerto act as a correction section configured to correct the direction ofthe predetermined axis set in the portable display device, by apredetermined angle, in a direction along which a display screen of theportable display device, which faces upward, stands up in a verticaldirection, the direction determination section determines whether or notthe direction of the predetermined axis, which is corrected by thecorrection section, is within the first range, the first object actioncontrol section controls the action of the first object in accordancewith the direction of the predetermined axis which is corrected by thecorrection section, and the second object action control sectioncontrols the action of the second object in accordance with thedirection of the predetermined axis which is corrected by the correctionsection.
 25. The computer-readable storage medium according to claim 1,wherein the first object action control section causes, as the firstobject, at least a target object which is a target of an attackaccording to an operation by a user, to move in the virtual world inaccordance with the predetermined axis, when the result of thedetermination is that the direction of the predetermined axis is withinthe first range, and the display control section includes a displayrange setting section configured to set a display range of the virtualworld to be displayed on the portable display device, based on theaction of the second object.
 26. The computer-readable storage mediumaccording to claim 25, wherein the display range setting section setsthe display range based on the action of the second object, by arranginga virtual camera for generating an image of the virtual world such thatthe first object and the second object are included in the displayrange, with a forward direction of the second object from behind thesecond object in the virtual world being a line-of-sight direction ofthe virtual camera, and the display control section displays, on theportable display device, as the first image, an image showing thevirtual world viewed from the virtual camera.
 27. The computer-readablestorage medium according to claim 25, wherein when the result of thedetermination is that the direction of the predetermined axis is withinthe first range, the first object action control section causes, as thefirst object, a shooting object which shoots a predetermined objecttoward the target to rotate in the virtual world in accordance with thedirection of the predetermined axis, and when the result of thedetermination is that the direction of the predetermined axis is outsidethe first range, the second object action control section causes thesecond object together with the shooting object to rotate in the virtualworld in accordance with the direction of the predetermined axis, andcauses the target object to move in the virtual world in accordance withthe direction of the shooting object.
 28. The computer-readable storagemedium according to claim 2, wherein the portable display deviceincludes at least one of a gyro sensor and an acceleration sensor, andthe attitude calculation section calculates an attitude of the portabledisplay device, based on data outputted from at least one of the gyrosensor and the acceleration sensor.
 29. The computer-readable storagemedium according to claim 1, wherein the information processing programfurther causes the computer to act as: a load obtaining sectionconfigured to obtain data based on a load applied to the load detectiondevice; and an object moving section configured to cause the firstobject and the second object to move in the direction of the secondobject in the virtual world, based on the data obtained by the loadobtaining section.
 30. The computer-readable storage medium according toclaim 29, wherein the information processing program further causes thecomputer to act as a center-of-gravity position calculation sectionconfigured to calculate a center-of-gravity position of the load appliedto the load detection device, based on the data obtained by the loadobtaining section, and the object moving section causes the first objectand the second object to move in the direction of the second object,based on the center-of-gravity position calculated by thecenter-of-gravity position calculation section.
 31. Thecomputer-readable storage medium according to claim 30, wherein theobject moving section sets a movement speed of the first object and thesecond object, based on a distance from a preset reference position tothe center-of-gravity position calculated by the center-of-gravityposition calculation section, and sets a forward movement or a backwardmovement of the first object and the second object, based on thedistance from the reference position to the center-of-gravity position,and causes the first object and the second object to move in thedirection of the second object.
 32. The computer-readable storage mediumaccording to claim 31, wherein the object moving section sets, as thereference position, a straight line which divides a part of the loaddetection device to which the load is applied, into a front part and arear part, sets a movement speed of the first object and the secondobject, based on a distance from the straight line to thecenter-of-gravity position calculated by the center-of-gravity positioncalculation section, sets the first object and the second object to aforward movement when the center-of-gravity position is located in thefront part with respect to the straight line, and sets the first objectand the second object to a backward movement when the center-of-gravityposition is located in the rear part with respect to the straight line.33. The computer-readable storage medium according to claim 1, whereinthe display control section outputs, to the portable display device,image data representing the first image, and the portable display deviceincludes: an image data obtaining section configured to obtain imagedata outputted from the information processing apparatus; and a displaysection configured to display the first image represented by the imagedata obtained by the image data obtaining section.
 34. Thecomputer-readable storage medium according to claim 33, wherein theinformation processing program further causes the computer to act as acompressed image generation section configured to generate compressedimage data by compressing the image data representing the first image,the display control section outputs the compressed image data generatedby the compressed image generation section to the portable displaydevice, the image data obtaining section obtains the compressed imagedata outputted from the information processing apparatus, the portabledisplay device further includes a display image decompression sectionconfigured to decompress the compressed image data to obtain image datarepresenting the first display image, and the display section displaysthe first image represented by the image data which has been obtained bythe image data obtaining section and decompressed by the display imagedecompression section.
 35. An information processing system including aportable display device, and an information processing apparatus capableof displaying an image on the portable display device, wherein theportable display device includes an attitude data output sectionconfigured to output, to the information processing apparatus, at leastattitude data based on an attitude of the portable display device, theinformation processing apparatus includes: a direction determinationsection configured to determine, based on the attitude data, whether ornot a direction of a predetermined axis set in the portable displaydevice is within a first range; a first object action control sectionconfigured to control an action of a first object arranged in a virtualworld, in accordance with the direction of the predetermined axis, whenthe result of the determination is that the direction of thepredetermined axis is within the first range; a second object actioncontrol section configured to control at least an action of a secondobject arranged in the virtual world, in accordance with the directionof the predetermined axis, when the result of the determination is thatthe direction of the predetermined axis is outside the first range; andan image data output section configured to output, to the portabledisplay device, a first image showing the virtual world including atleast the first object and the second object, and the portable displaydevice further includes: an image data obtaining section configured toobtain data of the first image outputted from the information processingapparatus; and a display section configured to display the first imagerepresented by the image data obtained by the image data obtainingsection.
 36. An information processing method executed by one processorincluded in an information processing system that includes at least oneinformation processing apparatus capable of displaying an image on aportable display device that outputs at least attitude data according toan attitude of a body of the portable display device, or by a pluralityof processors included in the information processing system in acooperative manner, the information processing method comprising thesteps of: determining, based on the attitude data, whether or not adirection of a predetermined axis set in the portable display device iswithin a first range; controlling an action of a first object arrangedin a virtual world, in accordance with the direction of thepredetermined axis, when the result of the determination is that thedirection of the predetermined axis is within the first range;controlling at least an action of a second object arranged in thevirtual world, in accordance with the direction of the predeterminedaxis, when the result of the determination is that the direction of thepredetermined axis is outside the first range; and displaying, on theportable display device, a first image showing the virtual worldincluding at least the first object and the second object.